Designer Projects and Sandboxes
In Data Shaper, a project is where multiple jobs or activities that relate to each other are grouped together.
Usually, a project corresponds to a use case, but in large organizations there might be one project per customer or per department (e.g., Finance or Marketing). There is no limit to the number of projects you can have.
Each new project in Data Shaper has a predefined structure. To get started, you can freely change the layout but we recommend not to upset it. In this way, you will help your fellow data engineers quickly understand the project if they are trying to learn.
Local projects
You’ll probably start by creating a local project. It will live just in Data Shaper Designer, and you run it on your machine as you’re building it.
Server projects
When you want to do something more complex or share your work with the team, you’ll typically deploy your project to a Data Shaper Server sandbox.
One project corresponds to one sandbox on the Server.
Why “sandbox” and not “project”?
A Server sandbox adds more than what you’ll find in a local project. You’ll get settings attached, e.g., access permissions which configure who has read/write access; runtime settings to configure if it can run in parallel/how many instances; and more.
Project life cycle
A project is created in Data Shaper Designer. When you want to run your project, you publish it to the Data Shaper Server. This is done directly from the Data Shaper Designer, simply right-clicking the project and selecting the Convert to Server Project context menu option and then selecting your Server.
The project then becomes connected to your Data Shaper Server sandbox. Your local Data Shaper Designer project lives locally, the Data Shaper Server sandbox lives wherever your Server is, but crucially when you make changes in Designer, they’re automatically performed in Server (both projects are synchronized, and the sync is two-way – changes made on Server will also show up in your local copy).
When you hit the run button, your graph runs on Server. But the Designer will keep tracking it and will show you the progress.
Updated about 1 year ago