Overview

The Job Inspector is a graphical tool, allowing authenticated users to view, track progress or investigate past executions from Data Shaper Server console. It is designed to help DataOps and DevOps team operate more efficiently in production environments, where using Data Shaper Designer may be undesirable or impossible.

The Job Inspector aims to provide tools necessary to check for configuration and data issues, helping support personnel to better understand processes and thus allowing them to create more accurate error reports for development teams.

It allows running jobs manually (see below) and setting their input parameters, which can be useful for troubleshooting.

By design, Job Inspector will never allow making any changes to any job of Data Shaper Server.

The tool is read-only, it does not allow editing of the displayed job.

Quickstart

The Job Inspector is available in the Sandboxes and Execution History sections.

Sandboxes

To open Job Inspector in Sandboxes, select a job and open the Job Inspector tab in the detail of the job.

Execution History

To open the Job Inspector in Execution History, select a job run from the list and open the Job Inspector tab in the detail of the run.

When the Job Inspector is opened from the Execution History, it contains not only job content but also information about the job run - numbers of records, component statuses and run execution log. Error message is available for failed components. It’s possible to inspect component’s configuration and metadata on the edges.

Using the Job Inspector

Job Panel

Is the main section of Job Inspector. Job panel visualizes data flow using Components and Edges. Its contents are responsive and allow basic interaction. The selection of any item on this panel will show its properties in the Detail panel. For larger transformations, it is possible to move the view using mouse drag action. Zoom in/out is via mouse wheel. Other panels can be opened or closed using the buttons below the Job panel.

Detail Panel

When there is an item active, the Detail panel is visible and shows different content based on the active item.

In general, the Detail panel shows the active item’s settings and properties. In the case of a Component that was terminated due to an error, it also shows an error message as a cause of failure.

Detail panel for components contains two lists:

• Configured properties - list of properties configured by a user
• Default properties - rest of the component properties with default values

Moreover, when component has an error status, error detail is displayed here.

Detail panel for edges contains info about metadata.

Log Panel

The Log panel is specific for jobs, currently running or executed in the past - i.e. only available when Job Inspector is opened from Execution History module.

It is possible to show / hide the Log panel using the button located below Job Inspector.

Log panel displays execution log, same as is available in a separate tab of Data Shaper Server’s Execution History module or Data Shaper Designer’s Console view.

Parameters Panel

Job parameters are key-value pairs used for job configuration. They allow executing the same job with different inputs, making the jobs reusable.

Parameters may help you understand what the job does. When a job fails, it is often necessary to find out which input caused the job to fail, so they are very useful for troubleshooting.

Job Inspector can display job parameters in one of the bottom panels. They are divided into three categories:

• Input parameters - parameters that were set when the job was executed
• Internal parameters - parameters stored internally in the current job file
• Linked parameters - parameters from linked .prm files

Outline Panel

Outline panel is a read-only panel that contains list of elements used in job and their properties. It is similar to the Outline pane in the Designer.

• Connections
• Sequences
• Lookups
• Dictionary

Clicking the "Outline" button opens the panel.

Data Inspector Panel

Like in Designer, Data Inspector can show sample data flowing through the selected edge of a running or finished job execution. By default, only the first 1000 records are saved for every edge.

The data is not available, unless the job is executed with data debugging enabled. Debug mode is enabled by default when the job is executed manually (see Running Jobs Manually below) from Sandboxes or Execution History, or using Designer.

Running Jobs Manually

In the Sandboxes Job Inspector you can run a job manually and observe its execution. The Job Inspector will show live component statuses and the numbers of records on edges of the running job. You can also set the input parameters for the execution.

In order to run a job, press the Run Graph button in the toolbar (indicated by the red arrow in the following figure). A dialog will appear, allowing you to set the input parameters, if necessary. If the server is suspended, the Run dialog requires explicit confirmation of execution on suspended server.

By default, the executed job will save edge data for viewing in Data Inspector (documented above). You can prevent it by unchecking the Enable Data Inspector checkbox.

Job Inspector in Separate Browser Tab

You can open the Job Inspector tab on a separate page outside of the Server Console via the Open in New Tabbutton located at the bottom. Once opened in a separate window, you can copy the URL and send it to a colleague.