WHAT IS COMPONENT PREVIEWER?
Component previewer is a new feature available in the Xplenty package designer, which allows you to preview your data at each transformation component as you build your package. Up until now, you could only preview data on some of our source components.
Apart from confirming the output of your transformation components, running a component preview is also a great way to identify issues within your package and your data flow logic.
The usual flow of creating packages and components does not change. Create and manage packages as you usually would, and if you feel the need to peek into your data pipeline, here are the steps:
- Open a package.
- Open or create a transformation component - it must receive data from other source or transformation components.
- Click the yellow Preview button and wait for the preview to complete. Do not close the component window while the previewer is running.
- The preview data is shown on the component once the previewer job is complete. Error messages could be displayed instead in case there are any errors.
The component previewer will provide real-time progress of the data preview job. At the end of the preview the result data will be shown in a table. However, sometimes the preview may contain an error, in which case the log box will show you the error message.
Below are a few examples of package errors during the component preview job.
Invalid source component
If the source component is not set up correctly, the preview will fail validation. In this example, the source connection is present but source properties and the schema section are not complete.
Running the previewer on a Select component attached to this source component returned the following error:
To fix this error, revisit the "file_storage_5" source component and set it up completely. Then re-run the previewer.
Transformation component produces no data
In this scenario, a transformation component in the preview path produces no data. For example, a Filter component might have filtered out all data.
Below are the logs which would be shown in this case.
The final message suggests to look through the components attached to the transformation component being previewed in order to identify any logic that might have caused no output of data.
Assertion error when using the Assert component
The Assert component allows you to ensure that data from a source complies with certain conditions before loading it into another component. If the source data doesn't meet these conditions, the job fails with an assertion failure. Similarly, when previewing an Assert component, the preview job will fail if conditions are not met.
An example below shows preview failure when a specified condition is not met by the source data.
The assertion message specified in the Assert component is visible when scrolling to the right end of the log box.
- Accounts using the component previewer feature must have a sufficient subscription level to allow for the creation of at least 1 sandbox cluster.
Component previewer is known to fail given the following conditions:
- Using BingAds or Google Adwords source connections
- BigQuery source data contains date
- BSON in source data (e.g. in MongoDB)
- RestAPI source component has package variable in request header field (e.g. Authorization $access_token)
As noted above, BingAds connections are currently not supported by component previewer.
Also, component preview data may not be consistent with standard job output since component previewer uses only sample data from the source connection. If the source connection has a wide range of values in a column, only a subset of those values might appear in the component previewer output.
An example using Aggregate component:
- Source data has 100 rows with a date column
- First 50 rows have date "1st July", next 50 rows have "3rd July"
- Previewer uses the first 50 rows as sample data - only "1st July" dates
- Component previewer on an Aggregate component which counts occurrence of each date will output
date | count
1st July | 50
- In a standard job, the output would be
date | count
1st July | 50
3rd July | 50