Bole Talent Network published "A Letter to Programmers About to Enter the Industry" on June 9, translated from "An open letter to those who want to start programming". My friend (his ID on this site is Mailper) told me that he hoped to see a more practical article on Coolshell. Since he is also a guy who loves programming and technology, I asked him to summarize some of his learning experiences with Python and web programming. He sent me some of his insights and experiences, which I made a few modifications to and added a "Advanced" section based on my own experiences. This is an article completed by a novice and me, an old guy, based on our experiences.
My friend named this article "Build Your Programming Technical Skills." I really didn't know how to translate it into Chinese, but while writing, I felt it resembled the process of leveling up in an online game, so I named it "Technical Leveling Guide." The title is a bit grand, haha, and it's purely for fun. This is just a sharing of Mailper's and my personal learning experiences. (Note: I have omitted some technologies I learned as a beginner (which are obviously outdated today), such as Delphi/Power Builder, and some technologies I found uninteresting like Lotus Notes/ActiveX/COM/ADO/ATL/.NET …)
Preface
Do you feel that when you graduated from school, you only worked on toy-like programs? Even if you have no experience after entering the workplace, you can go through the following extracurricular exercises (a friend's complaint: school courses always start from theory, and the homework projects seem to have no practical use; it's better to start from the needs in work).
Suggestions:
- Don't buy books randomly, don't chase new technologies and buzzwords; foundational knowledge accumulates over a long time and will be applicable for at least the next 10 years.
- Review history; by looking at the timeline of technological development, you can understand what tomorrow will look like.
- You must practice; no matter how simple the examples are, it's recommended to at least type them out yourself to see if you understand the details.
- You must learn to think; consider why it should be done this way and not that way. Also, think in a way that allows you to draw parallels.
- The current user interface is almost dominated by two things: 1) Web, 2) Mobile devices iOS or Android. The graphical interface of Windows is no longer popular.
- More and more companies are using cost-effective, high-performance Linux and various open-source technologies to build their systems; Windows is too costly.
- Microsoft's products change too quickly and are not durable; they are completely toying with programmers. For details, see "The History of the Windows Programming Revolution".
Enlightenment and Entry
1. Learn a scripting language, such as Python/Ruby
This can help you overcome the fear of low-level languages; scripting languages allow you to quickly develop usable small programs. Practical projects:
- Process text files or CSV (keywords: python csv, python open, python sys) read a local file and process it line by line (e.g., word count or log processing).
- Traverse the local file system (sys, os, path), for example, write a program to count the sizes of all files in a directory and sort them by various conditions and save the results.
- Interact with databases (python sqlite), write a small script to count the number of entries in a database.
- Learn to debug using various simple and straightforward methods like print.
- Learn to use Google (phrase, domain, use reader to follow tech blogs).
2. Get familiar with a programmer's editor (not an IDE) and some basic tools
- Vim / Emacs / Notepad++, learn how to configure code completion, appearance, external commands, etc.
- Source Insight (or ctag).
3. Familiarize yourself with Unix/Linux Shell and common command lines
- If you use Windows, at least learn to use Linux in a virtual machine; VMware Player is free, so install Ubuntu.
- Minimize the use of graphical interfaces.
- Learn to use man to view help.
- Understand the file system structure and basic operations: ls/chmod/chown/rm/find/ln/cat/mount/mkdir/tar/gzip …
- Learn to use some text manipulation commands: sed/awk/grep/tail/less/more …
- Learn to use some management commands: ps/top/lsof/netstat/kill/tcpdump/iptables/dd…
- Understand various configuration documents in the /etc directory, learn to view system logs in /var/log, and system runtime information in /proc.
- Understand regular expressions and use them to search for files.
4. Learn Web basics (HTML/CSS/JS) + server-side technology (LAMP)
The future will undoubtedly be a Web world, and the best website to learn Web basics is W3School.
- Learn the basic syntax of HTML.
- Learn how CSS selects HTML elements and applies some basic styles (keywords: box model).
- Learn to use Firefox + Firebug or Chrome to view the structure of web pages you find impressive and modify them dynamically.
- Learn to use JavaScript to manipulate HTML elements. Understand DOM and dynamic web pages (http://oreilly.com/catalog/9780596527402), there are free chapters online that are sufficient. Or refer to DOM.
- Learn to debug JavaScript code using Firefox + Firebug or Chrome (set breakpoints, view variables, performance, console, etc.).
- Configure Apache or Nginx on one machine.
- Learn PHP, enabling data interaction between backend PHP and frontend HTML, forming a preliminary understanding of server responses to browser requests. Implement a form submission and reflection function.
- Connect PHP to a local or remote MySQL database (MySQL and SQL are sufficient for learning on the spot).
- Complete a network programming course from a prestigious university (for example: http://www.stanford.edu/~ouster/cgi-bin/cs142-fall10/index.php). Don't think it requires more than one semester; college students take 3-5 courses full-time in one semester, and you can definitely keep up in your spare time.
- Learn a JavaScript library (such as jQuery or ExtJS) + Ajax (asynchronously load a server-side image or database content) + JSON data format.
- Read the first four chapters of HTTP: The Definitive Guide, and you will understand what happens when you browse the internet daily (proxy, gateway, browsers).
- Create a small website (for example: a small message board that supports user login, Cookie/Session, CRUD operations, image attachment uploads, and pagination).
- Buy a domain name, rent some space, and create your own website.
1. C Language and System Calls
- Relearn C language, understand pointers and memory models, implement various classic algorithms and data structures in C. Recommended books: "The Art of Computer Programming", "Introduction to Algorithms", and "Programming Pearls".
- Learn MIT Free Course) Introduction to Computer Science and Programming.
- Learn MIT Free Course) C Language Memory Management.
- Learn Unix/Linux system calls (Advanced Programming in the Unix Environment), understand system-level concepts.
- Use this system knowledge to manipulate the file system and users (implement a small program that can copy directory trees).
- Write a multi-process program using fork/wait/waitpid, and a multi-threaded program with synchronization or mutual exclusion using pthread. A multi-process ticket purchasing program.
- Implement a multi-process signal communication program using signal/kill/raise/alarm/pause/sigprocmask.
- Learn to use gcc and gdb for programming and debugging (refer to my "Debugging Programs with gdb").
- Learn to use makefile to compile programs. (Refer to my "Writing Makefile with Me").
- IPC and Socket topics can be practiced in advanced sections.
- Learn Windows SDK programming (Windows Programming).
- Write a window, understand WinMain/WinProcedure, and the Windows message mechanism.
- Write some programs to manipulate resource files or various graphical controls in the Windows SDK, as well as graphical programming.
- Learn how to use MSDN to view related SDK functions, various WM_ messages, and some examples.
- This book contains many examples; please do not copy them in practice, try to write your own examples.
- Don't worry too much about mastering these things, as GUI is being replaced by the Web; mainly understand programming for Windows graphical interfaces.
- The main learning of Java is to read the classic Core Java "Java Core Technology Programming" and "Thinking in Java" (there are two volumes; I only linked the first volume, which is sufficient, as understanding Java's graphical interface is enough).
- Learn JDK, learn to consult the Java API Doc http://download.oracle.com/javase/6/docs/api/.
- Understand the differences between Java as a virtual machine language and C and Python in terms of compilation and execution. Reflect on "cross-platform" technology from C, Java, and Python.
- Learn to use the IDE Eclipse, compile, debug, and develop Java programs using Eclipse.
- Set up a Tomcat website and try JSP/Servlet/JDBC/MySQL web development. Try to implement the previously mentioned PHP small project using JSP and Servlet.
- Learn HTML5; there are many tutorials online, and Coolshell has introduced many before, so I won't list them here.
- Learn about security issues in web development (refer to the attack on Sina Weibo and this article on Ruby).
- Learn the rewrite mechanism of HTTP Server, the reverse proxy mechanism of Nginx, and fast-cgi (e.g., PHP-FPM).
- Learn static page caching techniques for the web.
- Learn asynchronous workflow processing for the web, data caching, data partitioning, load balancing, and horizontal scaling architecture.
- Practical Tasks:
- Create some web animations using HTML5 canvas.
- Try SQL injection, JS injection, and XSS attacks on the previously developed web application.
- Modify the previously developed web application to be built on an Nginx + PHP-FPM + static page caching website.
- Learn to use SVN or Git to manage program versions.
- Learn to use JUnit for unit testing in Java.
- Learn the coding standards or coding guidelines for C and Java languages. (I wrote a very simple article about C language many years ago—"Programming Cultivation", you can find a lot of such things online).
- Recommended reading: "Code Complete", "Refactoring", "Clean Code".
1. C++ / Java and Object-Oriented Programming
I personally believe that mastering C++ makes learning Java a piece of cake. However, the learning curve for C++ is quite steep. Nevertheless, I think C++ is the language that needs to be mastered the most. Refer to two interesting articles: "C++ Learning Confidence Chart" and "21 Days to Master C++".
- Learn MIT Free Course) C++ Object-Oriented Programming.
- Read the books I recommended in my "How to Learn C++" at least twice (if you can deepen your understanding of C++ to the level of my articles "C++ Virtual Function Table Analysis" or "C++ Object Memory Storage (see above and below), or "C/C++ Traps in Returning Internal Static Members", that would be excellent).
- Then reflect on why C++ is designed this way, while Java is not? You must learn to compare the differences between C++ and Java, such as initialization, garbage collection, interfaces, exceptions, virtual functions, etc.
- Practical Tasks:
- Implement a BigInt in C++ that supports addition, subtraction, multiplication, and division of 128-bit integers.
- Encapsulate a data structure's capacity in C++, such as a hash table.
- Encapsulate and implement a smart pointer in C++ (make sure to use templates).
- Reading "Design Patterns" is essential; read it at least twice and think about the application scenarios of these 23 patterns. The main points are: 1) Prefer composition over inheritance, 2) Prefer interfaces over implementation. (Also recommend "Design Patterns Explained").
- Practical Tasks:
- Use the factory pattern to implement a memory pool.
- Use the strategy pattern to create a class that can left-align, right-align, and center-align text files.
- Use the command pattern to implement a command-line calculator that supports undo and redo.
- Use the decorator pattern to implement a pricing strategy for hotel room rates—peak season, service, VIP, tour groups, and other factors affecting prices.
- Learn the usage and design concepts of STL - containers, algorithms, iterators, function objects. If possible, read its source code.
- Practical Tasks: Try to use object-oriented programming, STL, design patterns, and Windows SDK graphical programming skills.
- Create a Snake or Tetris game that supports different levels and difficulties.
- Create a file browser that can browse files in a directory and perform different operations on different files; text files can be opened and edited, executable files can be executed, mp3 or avi files can be played, and image files can display images.
- Learn about the design of some C++ libraries, such as MFC (see Teacher Hou Jie's "Deep Dive into MFC"), Boost, ACE, CPPUnit, STL (STL might be too difficult, but if you can understand its design patterns and designs, that would be great; if you can delve into my article "Copy-on-Write Technique of STL String Class", that would be excellent; ACE requires strong system knowledge, refer to the later section "Strengthen Understanding of the System").
- Java is a truly object-oriented language, and it has an abundance of design patterns, making it the best language for learning object-oriented design patterns (see Design Patterns in Java).
- Recommended reading: "Effective Java" and "Java Concurrency in Practice".
- Learn Java frameworks; there are many Java frameworks, such as Spring, Hibernate, Struts, etc., mainly to learn Java design, such as IoC, etc.
- Java technologies are also numerous; focus on learning J2EE architecture and technologies like JMS, RMI, etc., for message passing and remote invocation.
- Learn to use Java to create Web Services (official tutorial here).
- Practical Tasks: Try to build a networked Web Service remote invocation program under the Spring or Hibernate framework, and enable message passing between two services using JMS.
- Dive deep into C++ (I have been delving into C/C++ for about ten years).
- Learn various design patterns in Java.
Important readings include the following books:
- "The Art of Unix Programming" to understand the design and development philosophy, cultural system, principles, and experiences in the Unix system domain. You will definitely feel enlightened.
- "Unix Network Programming Volume 1, Sockets" is a book that will help you understand network programming after reading. Pay special attention to the differences between TCP, UDP, and the multiplexing system calls select/poll/epoll.
- "TCP/IP Illustrated Volume 1: Protocols" - this is a book that will allow you to become a network hacker after reading. Understand how Ethernet operates, the TCP/IP protocols, their operating principles, and how to tune TCP.
- Practical Tasks:
- Understand what blocking (synchronous IO), non-blocking (asynchronous IO), and multiplexing (select, poll, epoll) IO technologies are.
- Write a network chat program with a chat server and multiple chat clients (the server uses UDP for multicast or broadcast to some or all chat clients).
- Write a simple HTTP server.
- "Unix Network Programming Volume 2, Inter-Process Communication" covers various IPC techniques like semaphores, pipes, shared memory, messages, etc. These techniques may seem a bit outdated, but they are still worth understanding.
- Practical Tasks:
- Mainly practice various IPC methods for inter-process communication.
- Try writing a pipe program where a parent and child process exchange data through a pipe.
- Try writing a shared memory program where two processes exchange a C structure array through shared memory.
- Study "Windows Core Programming". Master CreateProcess, Windows threads, thread scheduling, thread synchronization (Event, semaphore, mutex), asynchronous I/O, memory management, DLLs, and other major areas.
- Practical Tasks: Use CreateProcess to launch Notepad or IE and monitor the program's execution. Implement the previously written simple HTTP service using a thread pool. Write a DLL hook program to monitor the closing event of a specified window or record keystrokes in a certain window.
- With knowledge of multi-threading, multi-process communication, TCP/IP, sockets, C++, and design patterns, you can study ACE. Use ACE to rewrite the chat program and HTTP server mentioned above (with a thread pool).
- Practical Tasks: Using all the knowledge above, try to:
- Write a server that transmits large files to clients, requiring the bandwidth to be utilized at over 80% (note that disk I/O and network I/O may have issues; think about how to solve them; also, pay attention to the maximum transmission unit MTU).
- Understand the working principles of BT downloads and simulate the principles of BT downloads using a multi-process approach.
- Load balancing. HASH-based, purely dynamic. (You can search for some articles on load balancing on Google Scholar).
- Multi-layer distributed systems – client service node layer, computing node layer, data cache layer, data layer. J2EE is a classic multi-layer structure.
- CDN systems – nearby access, content edge computing.
- P2P systems, study the algorithms of BT and eMule. For example: DHT algorithm.
- Server backup, dual-machine backup systems (Live-Standby and Live-Live systems), how do two machines monitor each other through heartbeat? Cluster master node backup.
- Virtualization technology, using this technology, you can switch or reconfigure and deploy the operating system like an application.
- Learn Thrift, a binary high-performance communication middleware that supports data (object) serialization and various types of RPC services.
- Learn Hadoop. The core design of the Hadoop framework is MapReduce and HDFS. The idea of MapReduce is widely known from a paper by Google, and simply put, it is "task decomposition and result aggregation." HDFS stands for Hadoop Distributed File System, providing underlying support for distributed computing storage.
- Understand NoSQL databases (some say it might be an overhyped technology), but due to the increasing prevalence of ultra-large-scale and high-concurrency dynamic websites, and the rigid requirements for real-time data access in SNS-type websites, NoSQL databases are gradually becoming the focus of attention and are likely to replace relational databases as the mainstream data storage model in the future. Currently, there are many NoSQL databases, most of which are open-source, among which the more well-known ones include: MemcacheDB, Redis, Tokyo Cabinet (upgraded version is Kyoto Cabinet), Flare, MongoDB, CouchDB, Cassandra, Voldemort, etc.