Azure Storage Data Lake Service
Sends and receives files to/from Azure Data Lake Storage.
What’s inside
-
Azure Storage Data Lake Service component, URI syntax:
azure-storage-datalake:accountName/fileSystemName
Please refer to the above links for usage and configuration details.
Maven coordinates
<dependency>
<groupId>org.apache.camel.springboot</groupId>
<artifactId>camel-azure-storage-datalake-starter</artifactId>
</dependency>Spring Boot Auto-Configuration
The starter supports 39 options, which are listed below.
| Name | Description | Default | Type |
|---|---|---|---|
camel.component.azure-storage-datalake.account-key | account key for authentication | String | |
camel.component.azure-storage-datalake.autowired-enabled | Whether autowiring is enabled. This is used for automatic autowiring options (the option must be marked as autowired) by looking up in the registry to find if there is a single instance of matching type, which then gets configured on the component. This can be used for automatic configuring JDBC data sources, JMS connection factories, AWS Clients, etc. | true | Boolean |
camel.component.azure-storage-datalake.bridge-error-handler | Allows for bridging the consumer to the Camel routing Error Handler, which mean any exceptions (if possible) occurred while the Camel consumer is trying to pickup incoming messages, or the likes, will now be processed as a message and handled by the routing Error Handler. Important: This is only possible if the 3rd party component allows Camel to be alerted if an exception was thrown. Some components handle this internally only, and therefore bridgeErrorHandler is not possible. In other situations we may improve the Camel component to hook into the 3rd party component and make this possible for future releases. By default the consumer will use the org.apache.camel.spi.ExceptionHandler to deal with exceptions, that will be logged at WARN or ERROR level and ignored. | false | Boolean |
camel.component.azure-storage-datalake.client-id | client id for azure account | String | |
camel.component.azure-storage-datalake.client-secret | client secret for azure account | String | |
camel.component.azure-storage-datalake.client-secret-credential | client secret credential for authentication. The option is a com.azure.identity.ClientSecretCredential type. | ClientSecretCredential | |
camel.component.azure-storage-datalake.close | Whether or not a file changed event raised indicates completion (true) or modification (false) | false | Boolean |
camel.component.azure-storage-datalake.close-stream-after-read | check for closing stream after read | false | Boolean |
camel.component.azure-storage-datalake.configuration | configuration object for data lake. The option is a org.apache.camel.component.azure.storage.datalake.DataLakeConfiguration type. | DataLakeConfiguration | |
camel.component.azure-storage-datalake.credential-type | Determines the credential strategy to adopt | client-secret | CredentialType |
camel.component.azure-storage-datalake.data-count | count number of bytes to download | Long | |
camel.component.azure-storage-datalake.directory-name | directory of the file to be handled in component | String | |
camel.component.azure-storage-datalake.download-link-expiration | download link expiration time | Long | |
camel.component.azure-storage-datalake.enabled | Whether to enable auto configuration of the azure-storage-datalake component. This is enabled by default. | Boolean | |
camel.component.azure-storage-datalake.expression | expression for queryInputStream | String | |
camel.component.azure-storage-datalake.file-dir | directory of file to do operations in the local system | String | |
camel.component.azure-storage-datalake.file-name | name of file to be handled in component | String | |
camel.component.azure-storage-datalake.file-offset | offset position in file for different operations | Long | |
camel.component.azure-storage-datalake.health-check-consumer-enabled | Used for enabling or disabling all consumer based health checks from this component | true | Boolean |
camel.component.azure-storage-datalake.health-check-producer-enabled | Used for enabling or disabling all producer based health checks from this component. Notice: Camel has by default disabled all producer based health-checks. You can turn on producer checks globally by setting camel.health.producersEnabled=true. | true | Boolean |
camel.component.azure-storage-datalake.lazy-start-producer | Whether the producer should be started lazy (on the first message). By starting lazy you can use this to allow CamelContext and routes to startup in situations where a producer may otherwise fail during starting and cause the route to fail being started. By deferring this startup to be lazy then the startup failure can be handled during routing messages via Camel’s routing error handlers. Beware that when the first message is processed then creating and starting the producer may take a little time and prolong the total processing time of the processing. | false | Boolean |
camel.component.azure-storage-datalake.max-results | maximum number of results to show at a time | Integer | |
camel.component.azure-storage-datalake.max-retry-requests | no of retries to a given request | Integer | |
camel.component.azure-storage-datalake.open-options | set open options for creating file | OpenOption> | |
camel.component.azure-storage-datalake.operation | operation to be performed | listfilesystem | DataLakeOperationsDefinition |
camel.component.azure-storage-datalake.path | path in azure data lake for operations | String | |
camel.component.azure-storage-datalake.permission | permission string for the file | String | |
camel.component.azure-storage-datalake.position | This parameter allows the caller to upload data in parallel and control the order in which it is appended to the file. | Long | |
camel.component.azure-storage-datalake.recursive | recursively include all paths | false | Boolean |
camel.component.azure-storage-datalake.regex | regular expression for matching file names | String | |
camel.component.azure-storage-datalake.retain-uncommited-data | Whether or not uncommitted data is to be retained after the operation | false | Boolean |
camel.component.azure-storage-datalake.sas-credential | SAS token credential. The option is a com.azure.core.credential.AzureSasCredential type. | AzureSasCredential | |
camel.component.azure-storage-datalake.sas-signature | SAS token signature | String | |
camel.component.azure-storage-datalake.service-client | data lake service client for azure storage data lake. The option is a com.azure.storage.file.datalake.DataLakeServiceClient type. | DataLakeServiceClient | |
camel.component.azure-storage-datalake.shared-key-credential | shared key credential for azure data lake gen2. The option is a com.azure.storage.common.StorageSharedKeyCredential type. | StorageSharedKeyCredential | |
camel.component.azure-storage-datalake.tenant-id | tenant id for azure account | String | |
camel.component.azure-storage-datalake.timeout | Timeout for operation | Duration | |
camel.component.azure-storage-datalake.umask | umask permission for file | String | |
camel.component.azure-storage-datalake.user-principal-name-returned | whether or not to use upn | false | Boolean |