Azure Data Lake Reader Activity¶
Reads file from Azure Data Lake as a view.
Parameters¶
Name |
Description |
Type |
Default |
|---|---|---|---|
Title |
Title of the activity. Its displayed on the designer. |
text |
|
Path |
File path in adl[s]://<account>.azuredatalakestore.net/<path> format. |
text |
|
Format |
File format. See below for supported file formats. |
enum |
|
Schema |
Optional schema definition in Avro schema format. |
text |
|
As |
Name of DataRow view linked to the file. |
text |
|
Options |
Additional options (e.g. delimiter for CSV file could be specified as [delimiter,\t] if file is tab delimited |
key-value |
Supported File Formats¶
Name |
Description |
|---|---|
Parquet |
Apache Parquet is a columnar storage format. |
ORC |
ORC is also columnar storage format. |
Avro |
Apache Avro is a data serialization format. |
CSV |
CSV is a delimited text file format. |
JSON |
JSON is a lightweight data-interchange format. |
Job Parameters¶
Note
Following job parameters need to be set for proper Azure Data Lake authentication.
Key |
Value |
|---|---|
dfs.adls.oauth2.access.token.provider.type |
e.g. ClientCredential |
dfs.adls.oauth2.refresh.url |
under Azure Portal/Azure Active Directory/App Registrations/Endpoints/OAUTH 2.0 TOKEN ENDPOINT |
dfs.adls.oauth2.client.id |
AppId |
dfs.adls.oauth2.credential |
App Key |