You are a new engineer at a Databricks competitor startup named DataTricks, which is implementing its own Lakehouse platform. They have assigned you to the Admin Experience team, where you are building out setup and administration flows for the product.
Your PM and TL tell you that DataTricks wants to set up their Lakehouse platform as follows:
- A
Customerof DataTricks signs up for the product. - Each
Customercan have multipleUsers, as many different people at the company will want to use DataTricks (everyone from analysts to data engineers to ML practitioners). However, sinceUsersonly work for one company, eachUseronly belongs to a singleCustomer. - Each
Customercan have 0 or moreRepos, where aRepocontains all of your data science notebooks, BI dashboards, and ML assets. Further, aCustomercan assign 0 or moreUsersto any givenRepo, and eachUsercan belong to 0 or moreRepos. - A
Repois required to be associated with exactly 1ComputeEnvironment, which is a collection of cloud resources (a network for running Apache Spark, cloud storage for storing data, etc.) to use for data processing. OneComputeEnvironmentcan be shared by many differentRepos, but only from a single customer.
Part 1: Design the database schema
Please write up a database schema that can properly model the relationships above. Please write your thoughts below.
Part 2: ComputeEnvironment API
Now that you’ve designed the database, you are ready to split up the work amongst yourself and your team. You are in charge of the APIs to manage (e.g. Create / Read / Delete) ComputeEnvironments, and you leave user, repo, and customer management to your teammate. Let’s go ahead and design the APIs for ComputeEnvironments below.
Part 3: Dealing with slow clouds
You’ve started to implement your API, and you realize that CreateComputeEnv takes more than 10 minutes. Turns out creating cloud infrastructure is not very fast. You are frustrated, as when you make an API request, it takes forever to return, to the point that you wonder if it’s even really working. How do you update your system?
这是一道典型的多租户 SaaS 数据建模题,核心在于把 Customer、User、Repo、ComputeEnvironment 以及它们之间的一对多和多对多关系正确落到关系型数据库中。建模时通常会为 Customer、User、Repo、ComputeEnvironment 各建主表,再用中间表表示 User-Repo 的多对多关系,并通过外键约束保证 User 必须隶属于某个 Customer、Repo 必须且只能关联一个 ComputeEnvironment,同时确保 ComputeEnvironment 不能跨 Customer 共享。第二部分会进一步考察如何为 ComputeEnvironment 设计 CRUD API,第三部分则关注长耗时云资源创建的系统设计,通常需要把同步请求改成异步任务,并通过状态字段、轮询或回调来管理创建进度。