Skip to main content

System Design 7 Steps

Step 1: Requirement Clarifications
- Ask questions about the exact scope of the problem we are trying to solve.
- Open-ended

Step 2: Back-of-the envelope estimation
- Estimate the scale of the system we are going to design
- Focus Areas: scaling, partitioning, load balancing, caching

1) What scale is expected from the system? (e.g., number of new tweets, number of tweet views, number of timeline generations per second)

2) How much storage will we need? We will have different storage requirements if users can have photos and videos in their tweets

3) What network bandwidth usage are we expecting? This will be crucial in deciding how we will manage traffic and balance load between servers.

Step 3: System Interface Definition
- Define what APIs are expected from the system. This will establish the exact contract expected from the system and ensure if we haven't gotten any requirements wrong.

e.g., for twitter
postTweet(user_id, tweet_data, tweet_location, user_location, timestamp, …)  
generateTimeline(user_id, current_time, user_location, …)  
markTweetFavorite(user_id, tweet_id, timestamp, …)  

Step 4: Defining Data Model
- Defining the data model in the early part of the interview will clarify how data will flow between different system components. Later, it will guide for data partitioning and management. The candidate should identify various system entities, how they will interact with each other, and different aspects of data management like storage, transportation, encryption, etc.

e.g., for twitter
User: UserID, Name, Email, DoB, CreationDate, LastLogin, etc.
Tweet: TweetID, Content, TweetLocation, NumberOfLikes, TimeStamp, etc.
UserFollow: UserID1, UserID2
FavoriteTweets: UserID, TweetID, TimeStamp

Which database system should we use? Will NoSQL like Cassandra best fit our needs, or should we use a MySQL-like solution? What kind of block storage should we use to store photos and videos?

Step 5: High-level Design
- Draw a block diagram with 5-6 boxes representing the core components of our system. We should identify enough components that are needed to solve the actual problem from end to end.

- For Twitter, at a high level, we will need multiple application servers to serve all the read/write requests with load balancers in front of them for traffic distributions. If we assume we will have a lot more read traffic than write, we can decide to have seprate servers to handle these scenarios.

- On the back-end, we need an efficient database that can store all the tweets and support a large number of reads. We will also need a distributed file storage system for storing photos and videos.

Step 6: Detailed Design
- Dig deeper into two or three major components; The interviewer's feedback should always guide us to what parts of the system need further discussion. We should present different approaches, their pros and cons, and explain why we will prefer one approach over the other.

  • Since we will be storing a massive amount of data, how should we partition our data to distribute it to multiple databases? Should we try to store all the data of a user on the same database? What issue could it cause?
  • How will we handle hot users who tweet a lot or follow lots of people?
  • Since users’ timeline will contain the most recent (and relevant) tweets, should we try to store our data so that it is optimized for scanning the latest tweets?
  • How much and at which layer should we introduce cache to speed things up?
  • What components need better load balancing?


Step 7: Identifying and Resolving Bottlenecks
- Try to discuss as many bottlenecks as possible and different approaches to mitigate them.
  • Is there any single point of failure in our system? What are we doing to mitigate it?
  • Do we have enough replicas of the data so that we can still serve our users if we lose a few servers?
  • Similarly, do we have enough copies of different services running such that a few failures will not cause a total system shutdown?
  • How are we monitoring the performance of our service? Do we get alerts whenever critical components fail or their performance degrades?





Labels:

Comments

Post a Comment

Popular posts from this blog

Templates

  Template - Polymorphism is the ability of the same code to operate on different types. This ability to operate on multiple types reduces code duplication by allowing the same piece of code to be reused across the different types it can operate on. - Polymorphism comes in a variety of forms. What we are interested in at the moment is parametric polymorphism, meaning that we can write our code so that it is parameterized over what type it operates on.  -That is, we want to declare a type parameter T and replace int with T in the above code. -Then, when we want to call the function, we can specify the type for T and get the function we desire. C++ provides parametric polymorphism through templates. Templated Functions - We can write a templated function by using the keyword template followed by the template parameters in angle brackets (<>). - Unlike function parameters, template parameters may be types, which are specified with typename where the type of the parameter wo...
Post a Comment
Read more

前端 优化代码体积

当我使用npm run build的时候，项目构建了很久。所以考虑用create-react-app网站下面的工具来缩小代码体积  Analyzing the Bundle Size https://create-react-app.dev/docs/analyzing-the-bundle-size 更改完成后 npm run build npm run analyze 可以看到以下的图片: 其中main.js有1.57mb 然后起服务 serve -s build -l 8000 进入到首页之后，打开network，查看js，发现main.js有500kb。这个500kb是已经用gzip压缩过了，但是却还有这么大。500*3=1500说明源文件有1.5mb左右 其中, antd占了25%, recharts占了13%, react-dom占了7.6%，dnd-kit占了2.8% 其中recharts用于统计页面，dnd-kit用于拖拽排序-编辑器页面。 所以在加载首页的时候，先不加载编辑页面和统计页面的js的话，体积会小很多。 路由懒加载 因为项目中，体积占比最大的是Edit和Stat页面(编辑和统计页面)，所以考虑使用路由懒加载，拆分bundle，优化首页体积 router文件中，之前: import Edit from "../pages/question/Edit" ; import Stat from "../pages/question/Stat" ; 现在： const Edit = lazy (() => import ( "../pages/question/Edit" )); const Stat = lazy (() => import ( "../pages/question/Stat" )); 为了让生成的文件更加可读，可以改成下面这样: const Edit = lazy ( () => import ( /* webpackChunkName: "editPage" */ "../pages/question/Edit" ) ); const Stat ...
Post a Comment
Read more

useMemo的使用场景

 useMemo是用来缓存 计算属性 的。 计算属性是函数的返回值，或者说那些以返回一个值为目标的函数。 有些函数会需要我们手动去点击，有些函数是直接在渲染的时候就执行，在DOM区域被当作属性值一样去使用。后者被称为计算属性。 e.g. const Component = () => { const [ params1 , setParams1 ] = useState ( 0 ); const [ params2 , setParams2 ] = useState ( 0 ); //这种是需要我们手动去调用的函数 const handleFun1 = () => { console . log ( "我需要手动调用，你不点击我不执行" ); setParams1 (( val ) => val + 1 ); }; //这种被称为计算属性，不需要手动调用，在渲染阶段就会执行的。 const computedFun2 = () => { console . log ( "我又执行计算了" ); return params2 ; }; return ( < div onClick = { handleFun1 } > //每次重新渲染的时候我就会执行 computed: { computedFun2 () } </ div > ); }; 上面的代码中，在每次点击div的时候，因为setParams1的缘故，导致params1改变，整个组件都需要重新渲染，也导致comptedFunc2()也需要重新计算。如果computedFunc2的计算量很大，这时候重新计算会比较浪费。 可以使用useMemo: const Com = () => { const [ params1 , setParams1 ] = useState ( 0 ); const [ params2 , setParams2 ] = useState ( 0 ); //这种是需要我们手动去调用的函数 const handleFun1 ...
Post a Comment
Read more