Skip to main content

System Design 7 Steps

Step 1: Requirement Clarifications
- Ask questions about the exact scope of the problem we are trying to solve.
- Open-ended

Step 2: Back-of-the envelope estimation
- Estimate the scale of the system we are going to design
- Focus Areas: scaling, partitioning, load balancing, caching

1) What scale is expected from the system? (e.g., number of new tweets, number of tweet views, number of timeline generations per second)

2) How much storage will we need? We will have different storage requirements if users can have photos and videos in their tweets

3) What network bandwidth usage are we expecting? This will be crucial in deciding how we will manage traffic and balance load between servers.

Step 3: System Interface Definition
- Define what APIs are expected from the system. This will establish the exact contract expected from the system and ensure if we haven't gotten any requirements wrong.

e.g., for twitter
postTweet(user_id, tweet_data, tweet_location, user_location, timestamp, …)  
generateTimeline(user_id, current_time, user_location, …)  
markTweetFavorite(user_id, tweet_id, timestamp, …)  

Step 4: Defining Data Model
- Defining the data model in the early part of the interview will clarify how data will flow between different system components. Later, it will guide for data partitioning and management. The candidate should identify various system entities, how they will interact with each other, and different aspects of data management like storage, transportation, encryption, etc.

e.g., for twitter
User: UserID, Name, Email, DoB, CreationDate, LastLogin, etc.
Tweet: TweetID, Content, TweetLocation, NumberOfLikes, TimeStamp, etc.
UserFollow: UserID1, UserID2
FavoriteTweets: UserID, TweetID, TimeStamp

Which database system should we use? Will NoSQL like Cassandra best fit our needs, or should we use a MySQL-like solution? What kind of block storage should we use to store photos and videos?

Step 5: High-level Design
- Draw a block diagram with 5-6 boxes representing the core components of our system. We should identify enough components that are needed to solve the actual problem from end to end.

- For Twitter, at a high level, we will need multiple application servers to serve all the read/write requests with load balancers in front of them for traffic distributions. If we assume we will have a lot more read traffic than write, we can decide to have seprate servers to handle these scenarios.

- On the back-end, we need an efficient database that can store all the tweets and support a large number of reads. We will also need a distributed file storage system for storing photos and videos.

Step 6: Detailed Design
- Dig deeper into two or three major components; The interviewer's feedback should always guide us to what parts of the system need further discussion. We should present different approaches, their pros and cons, and explain why we will prefer one approach over the other.

  • Since we will be storing a massive amount of data, how should we partition our data to distribute it to multiple databases? Should we try to store all the data of a user on the same database? What issue could it cause?
  • How will we handle hot users who tweet a lot or follow lots of people?
  • Since users’ timeline will contain the most recent (and relevant) tweets, should we try to store our data so that it is optimized for scanning the latest tweets?
  • How much and at which layer should we introduce cache to speed things up?
  • What components need better load balancing?


Step 7: Identifying and Resolving Bottlenecks
- Try to discuss as many bottlenecks as possible and different approaches to mitigate them.
  • Is there any single point of failure in our system? What are we doing to mitigate it?
  • Do we have enough replicas of the data so that we can still serve our users if we lose a few servers?
  • Similarly, do we have enough copies of different services running such that a few failures will not cause a total system shutdown?
  • How are we monitoring the performance of our service? Do we get alerts whenever critical components fail or their performance degrades?





Labels:

Comments

Post a Comment

Popular posts from this blog

Templates

  Template - Polymorphism is the ability of the same code to operate on different types. This ability to operate on multiple types reduces code duplication by allowing the same piece of code to be reused across the different types it can operate on. - Polymorphism comes in a variety of forms. What we are interested in at the moment is parametric polymorphism, meaning that we can write our code so that it is parameterized over what type it operates on.  -That is, we want to declare a type parameter T and replace int with T in the above code. -Then, when we want to call the function, we can specify the type for T and get the function we desire. C++ provides parametric polymorphism through templates. Templated Functions - We can write a templated function by using the keyword template followed by the template parameters in angle brackets (<>). - Unlike function parameters, template parameters may be types, which are specified with typename where the type of the parameter wo...
Post a Comment
Read more

useMemo的使用场景

 useMemo是用来缓存 计算属性 的。 计算属性是函数的返回值，或者说那些以返回一个值为目标的函数。 有些函数会需要我们手动去点击，有些函数是直接在渲染的时候就执行，在DOM区域被当作属性值一样去使用。后者被称为计算属性。 e.g. const Component = () => { const [ params1 , setParams1 ] = useState ( 0 ); const [ params2 , setParams2 ] = useState ( 0 ); //这种是需要我们手动去调用的函数 const handleFun1 = () => { console . log ( "我需要手动调用，你不点击我不执行" ); setParams1 (( val ) => val + 1 ); }; //这种被称为计算属性，不需要手动调用，在渲染阶段就会执行的。 const computedFun2 = () => { console . log ( "我又执行计算了" ); return params2 ; }; return ( < div onClick = { handleFun1 } > //每次重新渲染的时候我就会执行 computed: { computedFun2 () } </ div > ); }; 上面的代码中，在每次点击div的时候，因为setParams1的缘故，导致params1改变，整个组件都需要重新渲染，也导致comptedFunc2()也需要重新计算。如果computedFunc2的计算量很大，这时候重新计算会比较浪费。 可以使用useMemo: const Com = () => { const [ params1 , setParams1 ] = useState ( 0 ); const [ params2 , setParams2 ] = useState ( 0 ); //这种是需要我们手动去调用的函数 const handleFun1 ...
Post a Comment
Read more

Valgrind

  Using Valgrind to check memory How to use Valgrind -To valgrind the program, run the valgrind command and give it our program name as an argument. -For example, if we want to run ./myProgram hello 42, we can simply run Valgrind ./myProgram hello 42.  Uninitialized Values -When we run the program, we may use uninitialized values. It needs to be fixed. Valgrind can tell us about the use of uninitialized values. But it only tell when the control flow of the program depends on the unitialized value. For example, uninitialized value appears in the conditional expression of an if, or a loop, or in the switch statement. -If we want to know where the uninitialized value is from, we can use Valgrind    --track-origins=yes ./myProgram -Using -fsanitize=address can find a lot of problems that Memcheck cannot.  -We can use Valgrind with GDB to debug. We can run: --vgdb=full --vgdb-error=0., then Valgrind will stop on the first error that it encounters and give control to ...
Post a Comment
Read more