First, let’s clarify what kind of distributed systems we’re discussing here. Simply put, they need to satisfy two conditions: multi-node and stateful. Multi-node is straightforward to understand. Being stateful means the system needs to maintain some data — otherwise, we could simply scale horizontally without limit, and there would be no distributed systems problem to speak of.

Common distributed systems — whether databases like MySQL, Cassandra, HBase, message queues like RocketMQ, Kafka, Pulsar, or infrastructure like ZooKeeper — all satisfy these two conditions.

The implementation of these distributed systems typically needs to address two aspects: first, the system’s own functionality; and second, maintaining good performance and stability in a distributed environment. Even two systems with completely different functions will have many similarities in how they address the second type of problem. This article focuses on the second type.

Core Challenges of Distributed Systems

Next, let’s enumerate the common goals of distributed systems, including but not limited to:

• A large number of ordinary servers interconnected via a network, collectively providing services to the outside world.
• As the cluster scales up, overall system performance scales linearly.
• Automatic fault tolerance — failed nodes are automatically migrated, and data consistency across different nodes is maintained.

What are the challenges in achieving these goals? Here are the main ones:

1. Process crashes: Caused by many factors, including hardware failures, software bugs, routine maintenance, and in cloud environments, more complex reasons. The biggest problem with process crashes is data loss. For performance reasons, we often don’t write synchronously to disk — instead, we temporarily buffer data in memory and periodically flush to disk. When a process crashes, the data in memory buffers is clearly lost.

2. Network delays and interruptions: When communication between nodes becomes very slow, how does one node determine whether another node is healthy?

3. Network partitions: The cluster’s nodes split into two subsets — communication within each subset works, but communication between subsets is broken (brain split). How should the cluster continue to serve requests?

   A brief digression: under the CAP theorem, real-world systems typically adopt one of two modes — CP mode (giving up high availability) or AP mode (giving up strong consistency). Why is there no CA mode that gives up partition tolerance? Because we cannot assume network communication will always be reliable. Once a cluster splits into two partitions, re-merging is not realistic.

4. Process pauses: For example, a full GC may cause a process to become temporarily unavailable before quickly recovering. During the unavailable period, the cluster may have already reacted accordingly. How should state consistency be maintained when the node comes back?

5. Clock skew and message reordering: We want operations across different nodes in the cluster to have a clear ordering. Clock skew between nodes prevents us from using timestamps to guarantee this. Message reordering further complicates distributed system processing.

Now, let’s walk through the solutions to these problems.

Partitioning and Replication

For process crashes, the key point is that simply ensuring no data loss during a crash is not difficult at all — the real challenge is achieving this while maintaining system performance.

The first pattern to introduce is Write-Ahead Log (WAL): the server stores every state change as a command in an append-only file on disk. Since append operations are sequential disk writes, they are typically very fast and can be completed without impacting performance. During server failure recovery, the log can be replayed to rebuild in-memory state.

The core idea is to first persist data in a low-cost way (not necessarily limited to sequential disk writes), then acknowledge the client without blocking other operations. The server asynchronously performs the subsequent high-cost work.

Typical scenarios and variants: MySQL redo log; Redis AOF; Kafka itself; common practices in application development — for time-consuming operations, first write a database record indicating the task will be executed, then perform the actual task asynchronously.

WAL introduces a minor issue: the log keeps growing. How to handle its own storage? Two natural approaches: segmentation and cleanup.

Segmentation splits a large log into smaller ones. Since WAL logic is generally simple, splitting is not complex — much easier than typical database sharding. This pattern is called Segmented Log, and the canonical implementation is Kafka’s partitions.

For cleanup, there is a pattern called low-water mark: a marker indicating which parts of the log can be cleaned up. The marking can be based on data status (redo log), preset retention time (Kafka), or more fine-grained cleanup and compaction (AOF).

Consistency Protocols

Moving to network-related issues, a simple heartbeat pattern solves inter-node state synchronization. If no heartbeat is received within a time window, the node is considered down.

For brain-split scenarios, the Quorum pattern is typically used. It requires the number of live nodes in the cluster to reach a Quorum value (typically, with 2f+1 nodes, at most f nodes can go down, so the Quorum value is f+1) before the cluster can serve requests. When examining implementations of distributed systems like RocketMQ or ZooKeeper, you’ll find the requirement for a minimum number of live nodes — this is exactly the Quorum pattern.

Quorum solves the data durability problem — successfully written data will not be lost even if nodes fail. However, Quorum alone cannot guarantee strong consistency, because data on different nodes may differ in time. When clients connect to different nodes, they may see different results. This can be solved through the Leader and Followers pattern, where one node is elected as the leader to coordinate data replication across nodes and determine which data is visible to clients.

The High-Water Mark pattern determines which data is visible to clients. Generally, after a write is completed on a Quorum number of follower nodes, the data can be marked as visible to clients. The line of completed replication is the high-water mark.

The Leader-Followers pattern is so widely used that examples are unnecessary. There are many distributed election algorithms — Bully, ZAB, Paxos, Raft, among others. Paxos is difficult both to understand and implement. Bully triggers frequent elections when nodes go up and down frequently. Raft, on the other hand, strikes a relatively balanced trade-off in terms of stability, implementation complexity, and adoption — it is the most widely used distributed election algorithm. For example, Elasticsearch replaced Bully with Raft in version 7.0; Kafka 2.8 switched from ZooKeeper’s ZAB protocol to Raft.

At this point, let’s summarize. Essentially, an operation on a distributed system can be概括为 the following steps:

1. Write to the leader’s Write-Ahead Log
2. Write to 1 follower’s WAL
3. Write to the leader’s data store
4. Write to 1 follower’s data store
5. Write to a Quorum of followers’ WALs
6. Write to a Quorum of followers’ data stores

The order of steps 2–5 is not fixed. The most important way a distributed system balances performance and stability is, in essence, deciding the order of these operations and when to return a success acknowledgment to the client. For example, MySQL’s synchronous replication, asynchronous replication, and semi-synchronous replication are classic examples of these different orderings.

Fault Detection and Recovery

For process pauses, the main problem scenario is: suppose the leader pauses. During the pause, a new leader is elected. When the original leader recovers, what should happen? The Generation Clock pattern addresses this — simply put, each leader is assigned a monotonically increasing generation number to indicate which “generation” of leader it is. Concepts like Raft’s “term” and ZAB’s “epoch” are implementations of this Generation Clock idea.

For clock skew, in a distributed environment, clock differences between nodes are inevitable. Under the Leader-Followers pattern, this problem is already largely minimized. Many systems choose to perform all operations on the leader, with leader-follower replication taking the form of replicating and replaying logs. In this way, clock issues generally don’t need to be considered. The only potential issue arises during leader failover, where the old leader and new leader might produce out-of-order data.

One solution to clock skew is to set up a dedicated synchronization service called an NTP service. However, this approach is not perfect — since it involves network operations, some error is inevitable. When relying on NTP to solve clock skew, the system design must tolerate very small timing errors.

Actually, beyond forcibly aligning clocks, there are simpler approaches to consider. First, let’s think about a question: do we really need to ensure messages are ordered by real-world physical time? Not really. What we need is a consistent, reproducible way to determine message order so that all nodes agree on the order. In other words, messages don’t have to follow physical chronological order, but the order determined by different nodes should be the same.

The Lamport Clock technique achieves this goal. Its logic is simple, as shown in the diagram:

Operations on a local node increment the local stamp by 1. During network communication — for example, when node C receives data from node B — it compares its current stamp with B’s stamp + 1 and takes the larger value as its new stamp. This simple operation guarantees that the order of any two causally related operations (whether on the same node or communicated between nodes) is consistent across all nodes.

Design Principles Summary

Additionally, there are relatively simpler considerations that frequently arise in distributed system design — such as how to distribute data evenly across nodes. For this, we might need to find an appropriate shard key based on the business scenario, or find a suitable hash algorithm. There is also consistent hashing technology, which gives us more flexibility in controlling data distribution.

Another key consideration in distributed system design is how to measure system performance, with metrics including latency, throughput, availability, consistency, and scalability. These are easy to understand conceptually, but measuring them more comprehensively — especially observing them conveniently — is itself a large topic.

Source: https://lichuanyang.top/en/posts/45718/

First, let’s sort out the necessary steps in the entire development process. I will briefly list them, including project creation, development, code review, deployment of test environment, deployment of grayscale and online environments, viewing online monitoring, and logs. , and troubleshooting problems, etc.

Based on this entire process chain, let’s take a look at what aspects we need to ensure in order to improve development and operation and maintenance efficiency as much as possible, and what low-cost solutions can be found in these aspects.

There are a few points:

• Quickly create a project and add necessary web, monitoring, log and other components;
• Improve the development efficiency of some conventional code development, such as database addition, deletion, modification and query;
• Simplify the process from packaging to deployment, and the test environment can be deployed automatically;
• A complete set of monitoring and alarms can be easily added to each service, including common machine, JVM, interface level indicators, and business-customized indicators;
• There is a place to centrally view the logs of each node;
• When necessary, powerful tools can be used on online nodes to help troubleshoot problems.

Next, let’s take a look at how to solve these problems respectively.

Deployment environment

First of all, in the title of this article, it is actually assumed that the deployment environment is kubernetes, so let me first talk about why kubernetes is the only option. Just flip through any introduction to kubernetes, and we can see many explanations of its advantages, so we won’t go into details here. The most critical one, and the benefit that everyone can experience personally, is that it provides a complete automatic expansion and contraction mechanism. It is very simple to set an expected value for CPU consumption, and the k8s cluster can adjust the number of pods based on the current CPU load. I compared the data before and after we enabled automatic expansion and contraction. The daily machine cost can differ by nearly 600 US dollars, accounting for more than 1/3 of the total cost.

Since the operation and maintenance cost of k8s itself is very high, it is recommended to directly purchase the services of a cloud service provider. Both AWS and Alibaba Cloud provide very complete k8s cluster functions.

In addition, it is recommended to use kuboard (https://kuboard.cn/) this tool to manage k8s. kuboard is a graphical k8s management tool, including common operations such as deployment, configuration, expansion, and logging in to pods, all of which can be operated on a graphical interface and is very convenient to use.

Project Development

Generally, you can create a project based on spring initializr. For common configurations such as logging and monitoring, it is recommended to compile a list of best practices and create a template project. New projects can be created based on this template project, using mvn archetype or similar tools to make the process smoother.

As for some commonly used basic codes such as databases and caches, I feel that Spring Family Bucket is easy enough to use.

For code review, we can use GitLab’s webhook to automatically send notifications to the project team when the code is submitted.

Packaging & Deployment

For springboot projects, you can use buildpacks to create docker images without having to consider the details of the docker file. For specific buildpacks, we use paketo buildpacks. Others such as cloudfoundry and heroku are similar. Currently, we have not studied the specific requirements for these packages.

Next, we can still use gitlab’s webhook to trigger a Jenkins build task. In the Jenkins build task, we can complete the operations of packaging and deploying to the kubernetes test cluster.

Monitoring & Alarming

It is recommended to use the prometheus + grafana package. Regarding the use of prometheus and the configuration under the springboot project, you can refer to my previous article h[ttps://lichuanyang.top/posts/28288/](https://lichuanyang. top/posts/28288/) .

On k8s, you can install a weave cloud agent, and then configure the automatic crawling of the prometheus interface.

In grafana, you can directly write promeql configuration monitoring reports. In addition, on the Grafana official website, there are a large number of charts shared by others that can be used directly.

In grafana, you can also configure alarms with various custom rules. If you use Feishu, you can easily use Feishu as the alarm channel for grafana by configuring grafana assistant in Feishu.

log

You can use loki (https://grafana.com/oss/loki/) as a tool for log collection and query. Loki can be considered as a lightweight ELK, and its maintenance cost will be much lower than ELK.

Troubleshooting

For java projects, using the artifact arthas can solve most problem troubleshooting needs. For arthas access, you can use arthas springboot starter. For pods on k8s, if the environment can be connected to the office environment, you can use the port forward function of k8s to forward the arthas port to the local. Of course, if you do this, be sure to control permissions.

Canary deployment

Regarding the role and implementation of canary deployment, you can refer to my other article https://lichuanyang.top/posts/30764/

To sum up, for operation and maintenance, you only need to maintain some infrastructure such as gitlab, kuboard, prometheus, grafana, loki, etc., and they are basically tools that are relatively simple to maintain. On this basis, with the help of reasonable processes and techniques, we can achieve a very good development experience.

Original address: https://lichuanyang.top/en/posts/40964/

Career Review

The biggest change in 2021 came from work. Since the industry I was in disappeared from the spotlight overnight, my plans to develop at one company for the long haul fell through, and I was forced to restart my career planning. A key consideration was whether to join a big company or a small company. The reason I mentioned reaching a critical crossroads is that at this age, whichever path I choose, there’s essentially no turning back. The capability requirements for a senior employee at big companies versus small companies are vastly different. At this stage, you no longer have the freedom to jump between big and small companies like you could in your early career after graduation.

To make this decision, I asked myself several questions:

Would I grow faster personally at a big company compared to a small one?

Would life at a big company be more comfortable than at a small one?

Will I need another big company experience as credentials in the future?

Could I earn more income at a big company?

After careful consideration, the answer to all these questions was actually no. Personal growth depends on the individual — I’m fundamentally not someone easily influenced by my environment. Regarding work-life balance, in theory it shouldn’t have much to do with company size, it just happens that several big companies in China aren’t great at this. On the credentials front, since I already have big tech experience, plus my educational background and other prior experience, I don’t think I’d lose out in that regard. Also, based on this job search and my experience interviewing candidates over the past two years, I’ve come to believe that the substance of your projects matters more than which company you were at. As for income, it depends on the macro environment, the company’s trajectory, the boss’s style, and luck — it generally doesn’t correlate much with company size.

After all this deliberation, I realized that the only advantage of joining a big tech company was that it would look good on paper. So I made my decision: completely abandon the big company idea and instead seek out promising startup companies with good culture and outlooks. Speaking of which, I’m sometimes deeply grateful for what seems like fate’s grace — it’s as if the universe always presents me with suitable options at key moments. In reality, though, I mainly have to thank the headhunter who provided information and support throughout the process, leading me to a new company that satisfied me on every front. I’ve been at the new company for almost four months now, and during this period, the room for doing meaningful work, personal growth, WLB, and financial returns have all exceeded expectations.

Life Changes

With the new work schedule, I also have more time with my daughter. The little girl has recently entered a language explosion phase, saying new words almost every day, and my sense of life happiness has reached a very high level.

Technical Growth

Additionally, I set some personal growth goals at the beginning of each year. At the start of 2021, my key goals were to expand and solidify my capability circle — striving to master new skills and reinforce existing ones. Specifically, this included improving technical and management skills related to my day job, enhancing business acumen and investment ability, and building some foundational skills.

On the professional front, I read Zhou Zhiming’s “Phoenix Architecture,” which played a key role in helping me build a more complete knowledge system and fill in some knowledge gaps. I studied Wang Zheng’s design patterns column on Geek Time, gained deeper understanding of design patterns, and gave a design patterns talk within my company. I also did a concentrated study of Elasticsearch, a technology I’d been using for a long time without seriously understanding.

On management, I learned some theoretical knowledge this year and have been working on applying it to practical work, with decent results.

On investments, I actually made some very mistaken attempts this year. Starting from early in the year, I tried to analyze company fundamentals myself to find suitable investment targets, reading many books and studying the experiences of notable investors. My current conclusion is that the barrier to this is extremely high — it’s unrealistic to achieve professional-level investment results through self-study alone. Based on this experience, my investment returns turned negative for the first time this year. Going forward, I’ll abandon individual stock investing and shift entirely to fund and index dollar-cost averaging. However, I won’t stop learning about business and investment knowledge, because I’ve realized that this knowledge can meaningfully transform one’s way of thinking, and the gains are well worth it.

Regarding foundational capabilities, I focused this year on cultivating what I call “clarity power.” This concept comes from the book “Cognitive Awakening” — the idea being that our fears are often rooted in “ambiguity,” meaning we haven’t clearly figured out what we’re actually facing. Once we have a clear understanding of a situation, we realize it’s not that difficult. The first time I encountered this concept, I immediately recognized that I’d always had a significant problem in this area, so I began deliberate practice. The results have been quite good — whether in work or life matters, I feel much more capable of handling things smoothly.

2022 Outlook

In summary, this year has been a year of critical decisions for me, and I’m steadily moving toward my life goals. I’m grateful to everyone who has helped me along the way. Here’s to continuing to give it my all in 2022.

Source: https://lichuanyang.top/en/posts/2345/

Frequently Asked Questions

Q: What’s the path forward for mid-career engineers?

The path doesn’t depend on company size — it depends on your capability circle. At the mid-to-late career stage, what matters more than “which company to join” is: what domain you have unique insights in, what level of technical problems you can solve, and whether your experience has crystallized into transferable methodologies. The author’s choice — letting go of big-tech prestige to seek startups with good culture and promising prospects — offers one viable path. But it’s not the only one. The key is to actively think about what you want, not passively wait for opportunities to land.

Q: How should technical people respond to industry changes?

In one phrase: expand your capability circle. An industry can vanish overnight (as the tutoring industry did for the author), but if you’ve been consistently broadening your capabilities, you gain the mobility to move across industries. Concretely: maintain technical sharpness (read seminal books like “Phoenix Architecture”), cultivate transferable foundational skills (like “clarity power”), and never anchor your sense of professional security to a single company or industry.

Q: Big company or startup — which is better for senior engineers?

It depends on what you need most at this stage. Big companies offer platform credentials and resource leverage, but personal growth pace and work-life balance aren’t guaranteed. Startups offer broader scope and faster decision-making, but stability is lower. The author’s conclusion after comparison: personal growth comes from yourself, not company size; if you already have big-tech experience, credentials matter less; and income depends more on the macro environment and luck than company size. The core advice: clarify your current-stage selection criteria before deciding, because at this stage of your career, there’s rarely a path back.

Design Thinking

An IP address is essentially a 32-bit integer. An “IP range” is simply a continuous range of integers. Given an IP, we need to find which range it falls into — a classic interval search problem.

The most direct approach is to iterate through all IP ranges — O(n) complexity. But there are hundreds of thousands of IP ranges globally, so iteration is too slow. A better approach is to sort the starting addresses of the IP ranges into an array and use binary search — O(log n) complexity, with only about 19 comparisons needed for hundreds of thousands of ranges.

Data Source

IP address database data can be obtained for free from IP2Location Lite. Sample data format:

1
2
3

16781312,JP
16785408,CN
16793600,JP

Each line contains two fields: the IP range’s starting address (converted to an integer) and the country code.

The original data also includes the end address of each IP range, but IP2Location’s data segments are continuous and gap-free — the end address of one range is exactly the starting address of the next range minus one. Therefore, we can store only the starting addresses, with the end address naturally determined by the next range’s starting address. This optimization saves half the memory.

Data Structure

1
2
3
4
5
6
7
8

public class IpCountryLookup {
    // IP range starting addresses (sorted)
    private final long[] startIps;
    // Corresponding country codes
    private final String[] countryCodes;
    // Total number of ranges
    private final int size;
}

Two parallel arrays: startIps[i] and countryCodes[i] correspond one-to-one. The binary search is performed on startIps, and once found, the result is taken from countryCodes.

Core Implementation: Binary Search

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29

public String lookup(String ip) {
    long ipLong = ipToLong(ip);
    
    // Binary search: find the last starting address <= ipLong
    int left = 0, right = size - 1;
    while (left <= right) {
        int mid = left + (right - left) / 2;
        if (startIps[mid] <= ipLong) {
            left = mid + 1;
        } else {
            right = mid - 1;
        }
    }
    
    // 'right' is the position of the last element <= ipLong
    if (right >= 0) {
        return countryCodes[right];
    }
    return "Unknown";
}

// Convert IP string to long
private long ipToLong(String ip) {
    String[] parts = ip.split("\\.");
    return (Long.parseLong(parts[0]) << 24)
         + (Long.parseLong(parts[1]) << 16)
         + (Long.parseLong(parts[2]) << 8)
         + Long.parseLong(parts[3]);
}

Performance Data

• Memory Usage: ~300,000 IP ranges, two arrays with ~300K elements each, total about 4 MB
• Query Latency: < 10 microseconds per query (~19 comparisons for binary search)
• QPS: Easily handles millions of queries per second on a single machine
• Data Updates: IP database updates roughly once a month; a scheduled task automatically pulls and reloads

Other Data Source Comparisons

If you only need country-level granularity, IP2Location Lite is sufficient. For city-level accuracy, MaxMind GeoLite2 is recommended.

Deployment

The project is built on Spring Boot and supports two deployment methods:

1
2
3
4
5

# Run directly
mvn spring-boot:run

# Docker
docker run -p 8080:8080 lcy362/ip-country

Try it online: http://ip-country.lichuanyang.top/

Project source: https://github.com/lcy362/ip-country

Source: https://lichuanyang.top/en/posts/36780/

In a typical development workflow, there are various review stages — product spec reviews, design reviews, test case reviews, code reviews, and so on. To conduct these reviews, meetings are inevitably introduced. Some people might think these activities are useless and a waste of time. In reality, they are extremely important. If you have the impression that reviews don’t matter, it’s likely due to one of two reasons: either a lack of understanding, or poor review practices in your environment.

Value of Review for the Team

Regarding the value of reviews, the most significant impact for a team is raising the team’s baseline. Imagine if no reviews were conducted at any stage — a product spec is handed off, developers just start coding, and any individual oversight could bring endless risks to the entire project. Multiple review stages essentially describe a business solution in different forms — product specs, code, test cases — and then involve more people to inspect each form, preventing individual oversights from creating business risks.

Furthermore, the focus of this article is to discuss the value of participating in reviews for an individual, beyond the benefits it brings to the team.

Value of Review for the Reviewee

First, for the person being reviewed — the author of the product spec, code, or other deliverables — a review is essentially a process where others help you improve. It’s a rare opportunity to receive direct feedback. For many people, compared to their student days, the biggest challenge of learning during their career is that no one “grades” their homework. After reading books or watching videos, there’s no way to confirm whether they’ve truly understood the material. Timely feedback, regardless of what you’re learning, is crucial. Among the reviewers will be your superiors and more capable colleagues. If they can thoroughly understand your solution or code, they can certainly provide valuable feedback. A review essentially gives you the opportunity to receive their feedback.

This implicitly requires the submitter to express their solution clearly, so others can participate in the review with minimal effort. For example, when writing code, use appropriate design patterns and write readable code. When writing documents, have a well-organized layout with proper sections and highlight key points. When presenting a solution to others, consider the audience’s knowledge background to ensure they can understand your proposal.

Another important aspect of reviews for the submitter is risk and responsibility sharing. Regardless of your company’s policies or your position, when a project you’re responsible for doesn’t go well, a significant portion of the consequences will ultimately fall on you. This may come as direct performance penalties, or as others questioning your capabilities — either way, it’s something nobody wants to face. Reviews transfer some of these consequences to the entire team. Of course, don’t think too much — the primary responsibility still remains with the submitter. This again highlights the importance of clear expression. The submitter needs to ensure that the key points of their solution are effectively communicated to the reviewers. If the reviewers successfully absorb this information, have thorough discussions during the review, and reach a consensus, that part of the decision can be considered as jointly made by the team.

Value of Review for the Reviewer

Now, let’s look at the value for reviewers — what does actively participating in others’ reviews offer? The primary benefit is a learning opportunity. As the saying goes, “When three people walk together, there must be one who can teach me.” From others’ solutions, you can certainly discover excellent points worth learning. When you find that someone else’s approach differs from your own expectations, it’s an opportunity for deep thinking — you can continuously question yourself to determine which approach is better until you reach an irrefutable conclusion. This kind of deep thinking significantly improves your own thinking methods.

Additionally, participating in reviews is also a way to rapidly gain experience. Reviewing is essentially spending review time to achieve an effect nearly identical to doing the work yourself. If you review thoroughly enough, you’ll gain a comprehensive understanding of how something is done, and this experience can fully count as part of your project experience.

Summary

This article comes to an end here. I hope you’ve gained an understanding of why reviews should be done and how to conduct them. Feel free to reach out to me through my WeChat official account (Mobility) or personal website.

Source: https://lichuanyang.top/en/posts/57205/

Trade-offs Across Four Dimensions

Career choice ultimately comes down to two key decisions: city and company. Choosing a city is highly subjective. For example, as someone from the north, I don’t like the climate in southern cities, I want to be close to home, and I need the opportunities that a big city offers — so Beijing was almost the only option. As practical advice, I’d suggest considering fast-growing strong second-tier cities like Chongqing or Hefei, where you can ride the wave of urban development and increase your own value along the way.

Within the scope of Beijing, I’ll discuss company choices based on my own experience.

Hukou: The Ticket to a Tier-1 City

When it comes to Beijing, one topic that inevitably comes up is the hukou (household registration). However, I strongly advise against letting a hukou decision dictate your entire life trajectory — for instance, joining a state-owned enterprise just for a hukou even though you have no desire to work in the system. Always prioritize thinking through your career development direction before worrying about hukou.

Many people agonize over public sector vs. private sector, big company vs. small company — but these are all surface-level choices. Even within the public sector, experiences can vary enormously. The right approach is to first clarify what you truly want, then seek out suitable opportunities. Knowing what you truly want isn’t easy for everyone, so let me break it down. Consider these questions:

• What do you most want from work? Money? Power? A sense of inner achievement? Or do you not expect anything from work at all?
• How much of your life do you want work to occupy?
• Among the resources gained through work (connections, information, etc.) and the capabilities developed, which do you feel more confident carrying into a different environment?
• Do you prefer doing routine, administrative work, or do you want to continuously tackle challenging problems?

For any of these questions, no answer is inherently better than another. Tackling difficult problems isn’t superior to doing routine work — it’s simply a choice based on personal values.

With any combination of these choices, you can find suitable opportunities. For instance, if you don’t expect much from work, don’t want it to dominate your life, and are willing to do routine work, some peripheral state-owned institutions could be a good fit. If you want power, are willing to let work take up significant time, and prefer to accumulate resources through work, consider core state-owned institutions with real authority. And if you want to earn money and prefer to develop personal capabilities, being a programmer is a solid choice.

Once you’ve thought through these questions, you can choose a company. Many times, we may not find a place that satisfies all our requirements. That’s okay — we can use job-hopping to address one part at a time, continuously gather information and improve our abilities, and find opportunities when we’re ready to seize them.

Big Tech: The Value and Cost of the Platform

Let me share how I approached my own key decisions based on my experience.

When I first graduated, my goal was to get a Beijing hukou while staying in the internet industry. I had two broad directions: competing for an SSP offer at an internet company — but my abilities at the time weren’t sufficient, so that path was hard to take. The other path was to find opportunities within state-owned enterprises where the business closely resembled internet scenarios, ideally consumer-facing products with large user bases and data volumes, so I wouldn’t fall too far behind industry-leading technologies. Fortunately, I found such an opportunity and successfully seized it. After joining, it was pretty much as I expected — although the company’s technical level wasn’t high and the corporate culture had many issues, the business scenarios were excellent and I had plenty of room to contribute. Through my work, I learned a great deal.

Next, I needed to address my big tech experience, so when I changed jobs, I only considered top-tier tech companies. The work itself was nothing special to speak of — although I wasn’t happy during that period, I achieved the goals I’d set, so there are no regrets.

For my next job change, I had three goals: significantly increase my income to close the gap with peers caused by my time at the state-owned enterprise; not work too much overtime; and have more relaxed interpersonal relationships. These three goals seem contradictory, don’t they? But with effort, I always managed to find suitable opportunities, and that’s how I ended up at my current company. Not all problems were solved perfectly, but they were addressed enough to make my work experience quite enjoyable.

Through my own experience, you can see that I always seemed to make contradictory demands — wanting both fish and bear paw — and fortunately, I managed to get both in the end. The truth is, there are so many companies in the world that no two conditions are necessarily mutually contradictory. There’s no need to give things up prematurely. For example, does being a programmer necessarily mean 996 work and giving up your life? Obviously not.

I’d like to address two more specific questions that arise from the above.

First, whether to choose a Beijing hukou. Let me first say what I gained from it. I caught the wave of property price surges in 2015-2016, which significantly increased my family’s assets even though I had no financial planning awareness at the time. This no longer holds true today — under the policy of housing for living, not speculation, even Beijing properties can’t outperform the stock market. Second, some procedures became slightly more convenient to handle, which I consider negligible — I probably don’t deal with paperwork more than once a year, and an increasing number of procedures support remote processing. Third, I can rest assured about my children’s education in Beijing without worrying about future uncertainty. I say “uncertainty” because I think it’s highly uncertain whether non-Beijing hukou holders will still be barred from taking the gaokao in Beijing nearly twenty years from now, and equally uncertain how competitive Beijing’s gaokao will become. A Beijing hukou essentially fixes my future choices, but what those choices will actually amount to is hard to predict.

So currently, the value of a Beijing hukou is quite limited. If I could go back to 2015, I’d still choose a Beijing hukou without hesitation. But if I had to make this choice today in 2021, I’d most likely decline it.

Second, regarding big tech experience. My view is that you must have it, and the sooner the better, but there’s no need to stay at a big company for life. The value of big tech experience has been thoroughly discussed by many, so I won’t repeat it here. Feel free to reach out if you’re interested in discussing further.

How to Choose

The above covers most of this article. One point I haven’t mentioned is the midlife crisis — some might wonder why. The reason is that I don’t think it’s a problem that can be solved through career choices alone. Some people think the public sector is stable — even if you coast every day, no one can touch you. But I don’t consider stability that depends on external conditions to be true stability. The public sector has had problems before in history. This path might give you a 99.99% chance of living peacefully for life, but if that 0.01% scenario occurs, what will you do? Being a programmer is different — perhaps only a 50% chance of staying comfortably at one company, but the capabilities developed through work give me the confidence to handle the other 50%. The only way to address the midlife crisis is self-improvement — whether building capabilities or accumulating resources, you must gain something from work to maintain your competitiveness.

I’m Liusha. You can connect with me through the WeChat public account (Mobility), my personal website, and other channels.

Source: https://lichuanyang.top/en/posts/34931/

Regarding the inputs and outputs of RPC interfaces, there has long been a school of thought that advocates wrapping all inputs and outputs into individual request and response classes. In this article, we will analyze whether this approach constitutes a sound design.

Input Parameter Encapsulation Analysis

Let’s start with inputs. I’ll state the conclusion first: encapsulation should only be done when there are excessive input parameters. In all other cases, there is absolutely no justification for encapsulation.

The reasons given for wrapping input parameters into a request class typically boil down to the following:

• Maintaining compatibility when interface parameters are modified
• Defining a generic abstract request class for unified management

First, the first point is a complete fallacy. The first time I encountered this claim, I was almost misled by it as well. But upon careful reflection, it becomes clear that this argument doesn’t hold at all.

Even with a request body approach, if the interface provider adds a mandatory field, the caller still needs to include this parameter to make the call. Compared to passing parameters directly, this doesn’t solve any problem. Furthermore, this kind of issue is fundamentally unsolvable because such situations should not arise in the first place. Interface providers have an obligation to ensure that subsequent modifications are forward-compatible. As for adding non-mandatory fields, even without wrapping in a request class, you can implement the new interface and maintain backward compatibility through an adapter pattern, and later asynchronously deprecate the old interface when the opportunity arises.

The second point does have some merit.

• For example, adding unified request parameters in an abstract class. I believe this is better handled in the framework layer rather than being exposed in business code.
• Another example is adding unified parameter validation. But if that’s the only purpose, using an overly abstract class is inappropriate. Instead, each related business domain should have its own abstract class, since only that domain would have such requirements. Moreover, even without wrapping into a request class, such requirements are not difficult to implement, as the difficulty itself doesn’t lie in non-uniform input structures.

The biggest problem with wrapping input parameters into a request class is that it destroys code readability and creates semantic ambiguity. A method like queryById(long id) has very clear semantics — a glance at the interface definition reveals the inputs and outputs. But after forced encapsulation, what inputs the interface actually requires becomes unclear. Additionally, if a request contains three parameters, can any combination of them return a result? This is also something you cannot determine just by looking at the interface. With normal parameter definitions, you can define a series of interfaces using the adapter pattern, making it easier for callers to understand.

The only condition for encapsulating input parameters is when there are too many parameters, and you need to use other means such as comments or documentation to specify the legal input values and upgrade strategies for the interface.

Output Result Encapsulation Analysis

Now let’s discuss outputs. Wrapping output results into a response body is somewhat more reasonable than wrapping input parameters, but it still shouldn’t be used indiscriminately. The general purpose of wrapping output results is to add a return status code layer.

Interface Segregation Principle Perspective

This essentially forces callers to understand the error conditions of the interface. We know that one of the fundamental design principles is the “Interface Segregation Principle,” which states that clients should not be forced to depend upon interfaces that they do not use. The situation here is analogous: if callers genuinely don’t care about the reasons for request success or failure, they shouldn’t be forced to understand them. For example, if I just want to query a count, there are only two possibilities — found or not found — and I don’t want to care about why it wasn’t found.

Therefore, when designing interfaces, you need to carefully consider whether the error conditions of an interface should be information that is provided externally. Only on this basis can you determine whether wrapping in a response object is appropriate or not.

Conclusion and Recommendations

In summary, blindly wrapping all input and output parameters is never advisable. You must carefully consider when encapsulation is appropriate and when it is not. If you have any thoughts on this topic, feel free to discuss them with me.

Source: https://lichuanyang.top/en/posts/20888/

Why I Don’t Do Infrastructure Development

First, I want to address a common misconception: that infrastructure development makes it easier to learn and grow faster. I believe this is nothing more than wishful thinking by business developers. Infrastructure development isn’t as conducive to learning as people think, and business development isn’t as detrimental to learning as people think — I’ll elaborate on the reasons below. The reason many people hold this belief is largely because infrastructure development positions typically have higher hiring standards, so these individuals naturally have better learning ability and motivation.

In reality, the daily work of infrastructure development isn’t that different from business development — both involve working on requirements. The only difference is that business requirements come from product managers, while infrastructure requirements come from other developers, or from the infrastructure developer’s own thinking and ideas. This brings me to my first reason for not wanting to do infrastructure development: the source of requirements is unscientific. If you only do infrastructure development and are detached from actual business scenarios, your understanding of technology is likely to be biased, and things you want to build may just be self-indulgent exercises with no practical application value. Even when collecting requirements from business developers, there are similar issues — their suggestions may reflect only shallow, unconsidered thoughts. Moreover, business developers may not accurately understand what infrastructure components should actually handle; they may not distinguish between what should be solved by infrastructure and what they should handle themselves.

This brings me to my second reason for not wanting to do infrastructure development: unclear boundaries of responsibility. This is closely tied to the capability of the business teams you work with. Sometimes, when something on your end isn’t done well, the business team finds a way to work around it. Other times, even when something has nothing to do with you, you’re forced to deal with some upper-layer logic.

As you can see, both issues above are extremely dependent on colleagues’ abilities. So if you insist on doing infrastructure development, you must go to a company with strong technical talent — this directly affects your work happiness. The company choice also impacts another aspect: the nature of the work. Simply put, it’s a difference between building wheels versus improving open-source tools — these two types of work are worlds apart. Building wheels requires deep familiarity with various fundamental knowledge and can indeed promote growth — this is the origin of the idea that “infrastructure development helps you grow.” But this kind of work only exists at a handful of big companies. The vast majority of infrastructure work — improving open-source tools — is no different from business development. You only need to learn the ins and outs of the open-source tools to complete most of the work, with neither depth nor breadth. This is why I said earlier that infrastructure development doesn’t help with learning.

So, to do infrastructure development, company selection is crucial — it must be large AND technically strong; both conditions are essential. You might ask: why not just go to such a company? That’s fair, but it brings its own problem — my third reason for not wanting to do infrastructure development: the career choice is extremely narrow. In China, there are only two companies doing Java infrastructure well, and everyone knows what their corporate cultures are like. So if you don’t want to go to those companies, does that mean you can’t do it at all? Even if you’re willing, the massive supply-demand imbalance would lead to insane involution, similar to algorithm positions. Do you have enough confidence to win that game? In contrast, for business development, big tech experience is just a necessary learning experience and a highlight on your resume. Once you’re familiar with the entire standardized workflow at a big company, you can freely choose wherever you want to go.

The Value of Business Development

I’ve talked too much about infrastructure development above. Let me now get to the main topic: the value of business development. In my view, business development has improved my capabilities primarily in three areas: accumulation of business knowledge, expansion of technical breadth, and improvement of comprehensive abilities.

First, business knowledge — business knowledge is essentially an understanding of the real world. For me personally, learning about how different industries operate in my spare time is first and foremost an interesting thing to do. Second, continuously learning industry knowledge across different domains has noticeably improved my cognitive abilities. Can this ability be monetized? Of course. For example, when investing in stocks, the accumulation from work has given me the ability to analyze a company’s business model — I can clearly understand what makes a good company and a good stock at a deeper level than most people. Of course, I’m getting ahead of myself here — take it as food for thought. The key is to follow your interest and see if you’re willing to learn about things beyond technology.

Next, technical breadth. The biggest difference between business development and infrastructure development is that infrastructure development demands more depth, while business development demands more breadth. Business development requires familiarity with a wide variety of frameworks and middleware — not just surface-level knowledge, but enough depth to understand what problems they solve, what problems they introduce, and which scenarios they’re suited for. You need to comprehensively understand complex business scenarios and complex systems, and be able to quickly locate problems when they occur. Beyond knowledge accumulation, this also requires experience gained from real-world scenarios.

Then there’s the improvement of comprehensive abilities. Compared to infrastructure development, business development exercises a person’s general foundational capabilities more — communication, problem analysis and problem solving, and the ability to weigh trade-offs. These abilities are related to technology, but not so closely. To be extreme, even if you change careers in the future, these universal capabilities will be very helpful. I often think about what the difference is between learning to use an open-source tool and playing Honor of Kings. As a beginner, following an example to build a demo is like encountering a game for the first time and just playing around. As you get more familiar, some people become emotionless tool-switching machines, while others gradually explore the underlying implementation. Just like in gaming — some people play thousands of matches without ever understanding the difference between Lightning Dagger and Infinity Edge, while others explore the numerical mechanics of damage. So I think we shouldn’t view programmers as some uniquely special profession — it’s fundamentally no different from other professions. Most professions are just solving problems in different ways: lawyers rely on law, fund managers rely on understanding markets and fundamentals, and programmers rely on code. What we continuously develop through work is our problem-solving ability, and no one requires the solution to be limited to technology alone.

In business development, we must continuously accumulate capabilities in these three areas, so that we can build momentum over time and confidently face the midlife crisis. Of course, I’m not writing this to say that business development is inherently better — I mainly want to help everyone clearly analyze the differences between business development and infrastructure development, so you can make a rational choice.

If you have any thoughts, feel free to reach out and discuss with me.

Source: https://lichuanyang.top/en/posts/40071/