> ## Documentation Index > Fetch the complete documentation index at: https://docs.lumos.com/llms.txt > Use this file to discover all available pages before exploring further. # /{connector_id}/get_data_recency > Retrieve the recency information for data surfaced by standard capabilities. This operation enables connectors to indicate how "fresh" the data is that they surface through their standard capabilities. This is particularly important for connectors that source data from scheduled dumps, snapshots, or other non-real-time data sources. ## Use Cases - **Data Freshness Assessment**: Understand how current the data is from each capability - **Scheduled Data Sources**: Handle connectors that read from CSV files, database snapshots, or other periodically updated data sources - **Data Quality Monitoring**: Track data staleness for compliance and accuracy ## Capability Recency Mapping The response maps each standard capability to an optional datetime indicating when the data for that capability was last updated: - **Non-null datetime**: Indicates the data for this capability is current as of the specified timestamp - **null datetime**: Indicates the connector cannot determine or does not track recency for this capability ## Connector Implementation Connectors should implement this capability when: - They source data from scheduled dumps or snapshots - They can provide meaningful recency information Connectors that always provide "live" data may return null for all capabilities or omit this capability entirely. ## Data Reconciliation ## OpenAPI ````yaml openapi/lumos-connectors.json POST /{connector_id}/get_data_recency openapi: 3.0.0 info: title: Lumos Connectors description: >- # The Lumos Connector API ## Introduction The Lumos Connector API is a standardized interface for Identity and Access Management (IAM) operations across various third-party systems. It enables seamless integration between Lumos and external applications by providing a consistent set of operations called **capabilities**. Each integration (referred to as a "connector") implements these capabilities to work with different third-party API providers, focusing primarily on: - User access management - License and cost tracking - User activity monitoring ## Core Components ### Connectors A connector is a specialized library that acts as a bridge between Lumos and third-party applications. It handles: - Translation of Lumos requests into app-specific API calls - Conversion of app-specific responses into standardized Lumos formats - Authentication and authorization flows - Data format transformations ### Capabilities Capabilities are standardized operations that each connector can implement. They provide: - Consistent interfaces across different connectors - Predictable behavior patterns - Standardized error handling - Unified data structures ## Data Model ### Accounts Accounts represent individual users or service accounts within a system. They serve as the primary entities for access management and support lifecycle operations such as creation, activation, deactivation, and deletion. Accounts can be associated with multiple entitlements and are typically identified by a unique account ID within the system. ### Entitlements Entitlements represent a permission or capability that can be granted to user accounts, such as a license or access level. They define specific permissions, access rights, or memberships and are always associated with a resource, which may be global or specific. Entitlements are categorized by `entitlement_type` (e.g., licenses, roles, permissions, group memberships) and have defined constraints for minimum and maximum assignments. The naming of entitlements may vary, such as using "membership" for group associations. ### Resources Resources represent entities within an application that can be accessed or managed. They are identified by a unique `resource_type` within each app and include a global resource (represented by an empty string) for top-level entities. Resources can represent hierarchical structures, such as Workspaces containing Users and Groups, and serve as the context for entitlement assignments. The usage of Resource IDs depends on the specific hierarchy, with an empty string for global resources and specific IDs (e.g., Workspace ID) for nested resources. ### Associations Associations define relationships from accounts to entitlements (which are resource specific). They follow a hierarchical structure of Account -> Entitlement -> Resource, with no direct account-to-resource associations allowed. Associations enable flexible access control models. Note: The specific structure and use of resources and entitlements may vary depending on the integrated system's architecture and access model. ## How to Use This API 1. Discover available connectors 2. Learn about a specific connector 3. Configure a connector 4. (optional) Authenticate with OAuth 5. Read data from the connected tenant 6. Write (update) data in the connected tenant ## Authenticating with a Connector ### Authentication Methods Connectors support two main authentication categories: ### 1. Shared Secret Authentication - API Keys / Tokens - Basic Authentication (username/password) ### 2. OAuth-based Authentication The API supports two OAuth flow types: #### Authorization Code Flow (3-legged OAuth) Requires a multi-step flow: 1. **Authorization URL** - Call `get_authorization_url` to start the OAuth flow - Redirect user to the returned authorization URL 2. **Handle Callback** - Process the OAuth callback using `handle_authorization_callback` - Receive access and refresh tokens 3. **Token Management** - Use `refresh_access_token` to maintain valid access - Store refresh tokens securely #### Client Credentials Flow (2-legged OAuth) Suitable for machine-to-machine authentication: 1. **Direct Token Request** - Call `handle_client_credentials_request` with client credentials - Receive access token (and optionally refresh token) 2. **Token Management** - Use `refresh_access_token` to maintain valid access (if refresh tokens are supported) - Store tokens securely The flow type is configured in the connector settings and determines which capabilities are available. Both flows support customizable authentication methods (Basic Auth or request body) and different request formats (JSON, form data, or query parameters). ### Validation After obtaining credentials: 1. Call `validate_credentials` to verify authentication 2. Retrieve the unique tenant ID for the authenticated organization ### Authentication Schema Each connector's `info.authentication_schema` defines: - Required credential fields - Field formats and constraints - OAuth scopes (if applicable) ## Pagination Lumos connectors implement a standardized pagination mechanism to handle large datasets efficiently. The pagination system uses opaque tokens to maintain state across requests. ### How Pagination Works 1. **Request Format** Every request can include an optional `page` parameter: ```typescript { "page": { "token": string, // Optional: opaque token from previous response "size": number // Optional: number of items per page } } ``` 2. **Response Format** Paginated responses include a `page` field: ```typescript { "response": T[], // Array of items "page": { "token": string, // Token for the next page "size": number // Items per page } } ``` ### Using Pagination 1. **Initial Request** - Make the first request without a page token - Optionally specify a page size 2. **Subsequent Requests** - Include the `token` from the previous response - Keep the same page size for consistency 3. **End of Data** - When there's no more data, the response won't include a page token ### Example Flow ```typescript // First request POST /connectors/pagerduty/list_accounts { "page": { "size": 100 } } // Response { "response": [...], "page": { "token": "eyJwYWdlIjogMn0=", "size": 100 } } // Next request POST /connectors/pagerduty/list_accounts { "page": { "token": "eyJwYWdlIjogMn0=", "size": 100 } } ``` ### Implementation Notes - Page tokens are opaque and should be treated as black boxes - Tokens may encode various information (page numbers, cursors, etc.) - The same page size should be used throughout a pagination sequence - Invalid or expired tokens will result in an error response version: 0.0.0 servers: [] security: [] tags: - name: Learning about connectors - name: Open API Specification - name: Read Capabilities - name: OAuth Capabilities - name: Write Capabilities paths: /{connector_id}/get_data_recency: post: tags: - Read Capabilities description: >- Retrieve the recency information for data surfaced by standard capabilities. This operation enables connectors to indicate how "fresh" the data is that they surface through their standard capabilities. This is particularly important for connectors that source data from scheduled dumps, snapshots, or other non-real-time data sources. ## Use Cases - **Data Freshness Assessment**: Understand how current the data is from each capability - **Scheduled Data Sources**: Handle connectors that read from CSV files, database snapshots, or other periodically updated data sources - **Data Quality Monitoring**: Track data staleness for compliance and accuracy ## Capability Recency Mapping The response maps each standard capability to an optional datetime indicating when the data for that capability was last updated: - **Non-null datetime**: Indicates the data for this capability is current as of the specified timestamp - **null datetime**: Indicates the connector cannot determine or does not track recency for this capability ## Connector Implementation Connectors should implement this capability when: - They source data from scheduled dumps or snapshots - They can provide meaningful recency information Connectors that always provide "live" data may return null for all capabilities or omit this capability entirely. ## Data Reconciliation operationId: get_data_recency parameters: - name: connector_id in: path required: true schema: type: string requestBody: required: true content: application/json: schema: type: object properties: auth: allOf: - $ref: '#/components/schemas/AuthCredential' description: The authentication credentials for the request. credentials: type: array items: $ref: '#/components/schemas/AuthCredential' allOf: - type: object required: - request properties: request: allOf: - $ref: '#/components/schemas/GetDataRecency' description: The main request payload. include_raw_data: type: boolean description: Whether to include raw data in the response. page: allOf: - $ref: '#/components/schemas/Page' description: Pagination information for the request. settings: type: object additionalProperties: {} description: >- Connector-specific settings for the request. These are settings that are shared across all capabilities. Usually contain additional required configuration options not specified by the capability schema. description: Generic request model. description: Authenticated request. responses: '200': description: The request has succeeded. content: application/json: schema: anyOf: - $ref: '#/components/schemas/GetDataRecencyResponse' - $ref: '#/components/schemas/ErrorResponse' components: schemas: AuthCredential: type: object properties: id: type: string description: The ID of the authentication schema. example: some_unique_id oauth: allOf: - $ref: '#/components/schemas/OAuthCredential' description: OAuth credentials, if using OAuth. oauth_client_credentials: allOf: - $ref: '#/components/schemas/OAuthClientCredential' description: OAuth client credentials, if using OAuth Client Credentials flow. oauth1: allOf: - $ref: '#/components/schemas/OAuth1Credential' description: OAuth 1.0a credentials, if using OAuth 1.0a. basic: allOf: - $ref: '#/components/schemas/BasicCredential' description: Basic auth credentials, if using basic auth. token: allOf: - $ref: '#/components/schemas/TokenCredential' description: Token credentials, if using token-based auth. jwt: allOf: - $ref: '#/components/schemas/JWTCredential' description: JWT credentials, if using JWT. service_account: allOf: - $ref: '#/components/schemas/ServiceAccountCredential' description: Service account credentials, if using service account. key_pair: allOf: - $ref: '#/components/schemas/KeyPairCredential' description: Key-pair credentials, if using key-pair auth. x-semantic: key-pair description: Authentication credentials, which can be one of several types. GetDataRecency: type: object allOf: - $ref: '#/components/schemas/EmptyButCreateAnyway' description: |- Request parameters for getting data recency information. This is a simple request model as the capability doesn't require any specific parameters - it returns recency information for all standard capabilities that the connector supports. x-capability-level: read Page: type: object properties: token: type: string description: |- Opaque token representing the next page of results. Include this token in subsequent requests to retrieve the next page. size: type: integer format: int32 description: |- Number of items to return per page. This should remain consistent throughout a pagination sequence. description: Pagination parameters for requests and responses. GetDataRecencyResponse: type: object allOf: - type: object required: - response properties: response: $ref: '#/components/schemas/DataRecency' raw_data: {} page: $ref: '#/components/schemas/Page' description: >- Response containing the main response payload, raw data, and pagination information. description: |- Response containing data recency information for standard capabilities. This response model wraps the DataRecency data and provides standard response metadata including success status and any error information. ErrorResponse: type: object required: - is_error - error properties: is_error: type: boolean description: Always true for error responses. default: true error: allOf: - $ref: '#/components/schemas/Error' description: The error details. raw_data: type: object additionalProperties: {} description: Raw data associated with the error, if any. page: allOf: - $ref: '#/components/schemas/Page' description: Pagination information, if applicable. description: Error details and metadata returned when an operation fails. OAuthCredential: type: object required: - access_token properties: access_token: type: string description: The OAuth access token. x-field_type: HIDDEN x-secret: true allOf: - $ref: '#/components/schemas/OAuthAuthentication' description: OAuth access token and related authentication data. x-credential-type: oauth OAuthClientCredential: type: object required: - access_token - client_id - client_secret - scopes properties: access_token: type: string description: The OAuth access token. x-field_type: HIDDEN x-secret: true client_id: type: string description: The OAuth client id. client_secret: type: string description: The OAuth client secret. x-secret: true x-field_type: SECRET scopes: type: array items: type: string description: The OAuth scopes list. description: OAuth Client Credentials x-credential-type: oauth_client_credentials OAuth1Credential: type: object required: - consumer_key - consumer_secret - token_id - token_secret properties: consumer_key: type: string consumer_secret: type: string token_id: type: string token_secret: type: string description: |- OAuth 1.0a credential model. This auth type is not used much, handling is done per-connector. x-credential-type: oauth1 BasicCredential: type: object required: - username - password properties: username: type: string description: The username for basic auth. password: type: string description: The password for basic auth. x-secret: true x-field_type: SECRET description: Basic authentication credentials. x-credential-type: basic TokenCredential: type: object required: - token properties: token: type: string description: The token for token-based auth. x-secret: true x-field_type: SECRET description: Token-based authentication credentials. x-credential-type: token JWTCredential: type: object required: - headers - claims - secret properties: headers: allOf: - $ref: '#/components/schemas/JWTHeaders' description: The JWT headers. claims: allOf: - $ref: '#/components/schemas/JWTClaims' description: The JWT claims. secret: type: string description: The JWT secret. description: JWT credential model. x-credential-type: jwt ServiceAccountCredential: type: object required: - key - impersonation_email - tenant_id - scopes properties: service_type: allOf: - $ref: '#/components/schemas/ServiceAccountType' description: The type of service. x-field_type: HIDDEN default: google_cloud key: type: object additionalProperties: {} description: The JSON key file contents for the service account x-field_type: HIDDEN x-secret: true impersonation_email: type: string description: The email of the user to impersonate. tenant_id: type: string description: Tenant ID scopes: type: array items: type: string description: |- The scopes to request. Requested by connector. Define a default here. x-field_type: HIDDEN x-credential-type: service_account KeyPairCredential: type: object required: - key_identifier - private_key properties: key_identifier: type: string description: The identifier for the key-pair. x-field_type: HIDDEN private_key: type: string description: The private key used to authenticate with the server. x-field_type: HIDDEN x-secret: true description: Key-pair credential model. x-keygen-type: SSH x-semantic: key-pair x-credential-type: key_pair EmptyButCreateAnyway: type: object description: >- This is a hack to force creation of types that extend this model, that otherwise don't have any properties but we still want the Python types for. DataRecency: type: object required: - data properties: data: type: object additionalProperties: type: string format: date-time nullable: true description: >- Map of standard capability names to their data recency timestamps. Each entry in this map represents a standard capability and its associated data recency information: - **Key**: The name of a standard capability (e.g., "list_accounts", "list_entitlements") - **Value**: Datetime indicating when the data for this capability was last updated (may be null if recency information is not available) ## Capability Coverage The map should include entries for all standard capabilities that the connector supports and for which it can provide meaningful recency information. ## Data Freshness Interpretation - **non-null timestamp**: Data is current as of the specified time - **null value**: Connector cannot determine due to an unexpected error - **missing key**: If the a StandardCapabilityName doesn't have a key in the map, then the connector doesn't support surfacing data recency for that capability. ## Common Use Cases - **list_accounts**: When the account data was last synchronized - **list_entitlements**: When permission/role data was last updated - **list_updated_accounts**: When the change tracking data was last refreshed description: >- Response model containing data recency information for standard capabilities. This model maps each standard capability to an optional datetime indicating when the data for that capability was last updated or refreshed. Error: type: object required: - message - error_code - app_id properties: message: type: string description: A human-readable string explaining what went wrong. example: Authentication credentials didn't work status_code: type: integer description: |- An http status code (if applicable) corresponding to the underlying app's HTTP response that triggered this error. example: 401 app_error_code: type: string description: |- An application-specific code string that can tell us more details about the underlying application code or request that triggered the error. example: my_integration.token_expired error_code: allOf: - $ref: '#/components/schemas/ErrorCode' description: >- A standard error code used by Lumos callers to respond to this error. example: unauthenticated app_id: type: string description: The ID of the app where the error occurred. example: pagerduty raised_by: type: string description: What type of exception triggered this error (if any) example: ConnectorError raised_in: type: string description: Where the exception was raised example: my_integration.integration:list_accounts description: An error that occurred during an operation. OAuthAuthentication: type: object required: - access_token properties: access_token: type: string description: The OAuth access token. x-field_type: HIDDEN x-secret: true description: |- OAuth authentication model. This is a model exclusively used by standard capabilities. x-credential-type: oauth JWTHeaders: type: object required: - alg - jku - jwk - typ - kid - x5u - x5c - x5t - x5t#S256 - cty - crit properties: alg: type: string description: The JWT algorithm. jku: type: string description: JWK Set URL. jwk: type: string description: JWK. typ: type: string description: The JWT type. kid: type: string description: The JWT key ID. x5u: type: string description: The X509 URL. x5c: type: string description: The X509 certificate chain. x5t: type: string description: The X509 certificate SHA-1 thumbprint. x5t#S256: type: string description: The X509 certificate SHA-256 thumbprint. cty: type: string description: The content type of the JWT. crit: type: array items: type: string description: The JWT critical extension. description: JWT headers model. JWTClaims: type: object required: - iss - sub - aud - exp - nbf - iat - jti - act - scope - client_id - may_act properties: iss: type: string description: The issuer of the JWT. sub: type: string description: The subject of the JWT. aud: type: string description: The audience of the JWT. exp: type: integer format: int64 description: The expiration time of the JWT in seconds since the Unix epoch. nbf: type: integer format: int64 description: The not before time of the JWT in seconds since the Unix epoch. iat: type: integer format: int64 description: The issue time of the JWT in seconds since the Unix epoch. jti: type: string description: The JWT ID. act: type: string description: The Actor of the JWT. scope: type: array items: type: string description: Scopes granted to the JWT. client_id: type: string description: The client ID of the JWT. may_act: type: string description: The may_act of the JWT. description: JWT payload model representing the claims in the JWT. ServiceAccountType: type: string enum: - google_cloud - google_drive - aws ErrorCode: type: string enum: - not_found - internal_error - api_error - unauthorized - bad_request - permission_denied - not_implemented - unexpected_error - unsupported_operation - unknown_value - invalid_value - bad_gateway - unauthenticated - request_timeout - connection_timeout - connection_rejected - rate_limit - authentication_expired - service_error - invalid_response - client_call_error - connection_closed - creds_revoked - invalid_page_token - integration_missing_parameter description: >- Enum representing various error codes that can be returned by the connector. ````