List

API Reference

Knowledge Bases

Data Sources

List Data Sources for a Knowledge Base

knowledge_bases.data_sources.list(strknowledge_base_uuid, DataSourceListParams**kwargs) -> DataSourceListResponse

get/v2/gen-ai/knowledge_bases/{knowledge_base_uuid}/data_sources

To list all data sources for a knowledge base, send a GET request to /v2/gen-ai/knowledge_bases/{knowledge_base_uuid}/data_sources.

ParametersExpand Collapse

knowledge_base_uuid: str

page: Optional[int]

Page number.

per_page: Optional[int]

Items per page.

ReturnsExpand Collapse

class DataSourceListResponse: …

A list of knowledge base data sources

knowledge_base_data_sources: Optional[List[APIKnowledgeBaseDataSource]]

The data sources

aws_data_source: Optional[AwsDataSource]

AWS S3 Data Source for Display

bucket_name: Optional[str]

Spaces bucket name

item_path: Optional[str]

region: Optional[str]

Region of bucket

bucket_name: Optional[str]

Name of storage bucket - Deprecated, moved to data_source_details

created_at: Optional[datetime]

Creation date / time

formatdate-time

dropbox_data_source: Optional[DropboxDataSource]

Dropbox Data Source for Display

folder: Optional[str]

file_upload_data_source: Optional[APIFileUploadDataSource]

File to upload as data source for knowledge base.

original_file_name: Optional[str]

The original file name

size_in_bytes: Optional[str]

The size of the file in bytes

formatuint64

stored_object_key: Optional[str]

The object key the file was stored as

item_path: Optional[str]

Path of folder or object in bucket - Deprecated, moved to data_source_details

last_datasource_indexing_job: Optional[APIIndexedDataSource]

completed_at: Optional[datetime]

Timestamp when data source completed indexing

formatdate-time

data_source_uuid: Optional[str]

Uuid of the indexed data source

error_details: Optional[str]

A detailed error description

error_msg: Optional[str]

A string code provinding a hint which part of the system experienced an error

failed_item_count: Optional[str]

Total count of files that have failed

formatuint64

indexed_file_count: Optional[str]

Total count of files that have been indexed

formatuint64

indexed_item_count: Optional[str]

Total count of files that have been indexed

formatuint64

removed_item_count: Optional[str]

Total count of files that have been removed

formatuint64

skipped_item_count: Optional[str]

Total count of files that have been skipped

formatuint64

started_at: Optional[datetime]

Timestamp when data source started indexing

formatdate-time

status: Optional[Literal["DATA_SOURCE_STATUS_UNKNOWN", "DATA_SOURCE_STATUS_IN_PROGRESS", "DATA_SOURCE_STATUS_UPDATED", 3 more]]

Accepts one of the following:

"DATA_SOURCE_STATUS_UNKNOWN"

"DATA_SOURCE_STATUS_IN_PROGRESS"

"DATA_SOURCE_STATUS_UPDATED"

"DATA_SOURCE_STATUS_PARTIALLY_UPDATED"

"DATA_SOURCE_STATUS_NOT_UPDATED"

"DATA_SOURCE_STATUS_FAILED"

total_bytes: Optional[str]

Total size of files in data source in bytes

formatuint64

total_bytes_indexed: Optional[str]

Total size of files in data source in bytes that have been indexed

formatuint64

total_file_count: Optional[str]

Total file count in the data source

formatuint64

last_indexing_job: Optional[APIIndexingJob]

IndexingJob description

completed_datasources: Optional[int]

Number of datasources indexed completed

formatint64

created_at: Optional[datetime]

Creation date / time

formatdate-time

data_source_uuids: Optional[List[str]]

finished_at: Optional[datetime]

formatdate-time

knowledge_base_uuid: Optional[str]

Knowledge base id

phase: Optional[Literal["BATCH_JOB_PHASE_UNKNOWN", "BATCH_JOB_PHASE_PENDING", "BATCH_JOB_PHASE_RUNNING", 4 more]]

Accepts one of the following:

"BATCH_JOB_PHASE_UNKNOWN"

"BATCH_JOB_PHASE_PENDING"

"BATCH_JOB_PHASE_RUNNING"

"BATCH_JOB_PHASE_SUCCEEDED"

"BATCH_JOB_PHASE_FAILED"

"BATCH_JOB_PHASE_ERROR"

"BATCH_JOB_PHASE_CANCELLED"

started_at: Optional[datetime]

formatdate-time

status: Optional[Literal["INDEX_JOB_STATUS_UNKNOWN", "INDEX_JOB_STATUS_PARTIAL", "INDEX_JOB_STATUS_IN_PROGRESS", 4 more]]

Accepts one of the following:

"INDEX_JOB_STATUS_UNKNOWN"

"INDEX_JOB_STATUS_PARTIAL"

"INDEX_JOB_STATUS_IN_PROGRESS"

"INDEX_JOB_STATUS_COMPLETED"

"INDEX_JOB_STATUS_FAILED"

"INDEX_JOB_STATUS_NO_CHANGES"

"INDEX_JOB_STATUS_PENDING"

tokens: Optional[int]

Number of tokens

formatint64

total_datasources: Optional[int]

Number of datasources being indexed

formatint64

total_items_failed: Optional[str]

Total Items Failed

formatuint64

total_items_indexed: Optional[str]

Total Items Indexed

formatuint64

total_items_skipped: Optional[str]

Total Items Skipped

formatuint64

updated_at: Optional[datetime]

Last modified

formatdate-time

uuid: Optional[str]

Unique id

region: Optional[str]

Region code - Deprecated, moved to data_source_details

spaces_data_source: Optional[APISpacesDataSource]

Spaces Bucket Data Source

bucket_name: Optional[str]

Spaces bucket name

item_path: Optional[str]

region: Optional[str]

Region of bucket

updated_at: Optional[datetime]

Last modified

formatdate-time

uuid: Optional[str]

Unique id of knowledge base

web_crawler_data_source: Optional[APIWebCrawlerDataSource]

WebCrawlerDataSource

base_url: Optional[str]

The base url to crawl.

crawling_option: Optional[Literal["UNKNOWN", "SCOPED", "PATH", 2 more]]

Options for specifying how URLs found on pages should be handled.

• UNKNOWN: Default unknown value
• SCOPED: Only include the base URL.
• PATH: Crawl the base URL and linked pages within the URL path.
• DOMAIN: Crawl the base URL and linked pages within the same domain.
• SUBDOMAINS: Crawl the base URL and linked pages for any subdomain.

Accepts one of the following:

"UNKNOWN"

"SCOPED"

"PATH"

"DOMAIN"

"SUBDOMAINS"

embed_media: Optional[bool]

Whether to ingest and index media (images, etc.) on web pages.

links: Optional[APILinks]

Links to other pages

pages: Optional[Pages]

Information about how to reach other pages

first: Optional[str]

First page

last: Optional[str]

Last page

next: Optional[str]

Next page

previous: Optional[str]

Previous page

meta: Optional[APIMeta]

Meta information about the data set

page: Optional[int]

The current page

formatint64

pages: Optional[int]

Total number of pages

formatint64

total: Optional[int]

Total amount of items over all pages

formatint64

List Data Sources for a Knowledge Base

python

• http
• typescript
• python

from gradient import Gradient

client = Gradient(
    access_token="My Access Token",
)
data_sources = client.knowledge_bases.data_sources.list(
    knowledge_base_uuid="\"123e4567-e89b-12d3-a456-426614174000\"",
)
print(data_sources.knowledge_base_data_sources)

200 example

{
  "knowledge_base_data_sources": [
    {
      "aws_data_source": {
        "bucket_name": "example name",
        "item_path": "example string",
        "region": "example string"
      },
      "bucket_name": "example name",
      "created_at": "2023-01-01T00:00:00Z",
      "dropbox_data_source": {
        "folder": "example string"
      },
      "file_upload_data_source": {
        "original_file_name": "example name",
        "size_in_bytes": "12345",
        "stored_object_key": "example string"
      },
      "item_path": "example string",
      "last_datasource_indexing_job": {
        "completed_at": "2023-01-01T00:00:00Z",
        "data_source_uuid": "123e4567-e89b-12d3-a456-426614174000",
        "error_details": "example string",
        "error_msg": "example string",
        "failed_item_count": "12345",
        "indexed_file_count": "12345",
        "indexed_item_count": "12345",
        "removed_item_count": "12345",
        "skipped_item_count": "12345",
        "started_at": "2023-01-01T00:00:00Z",
        "status": "DATA_SOURCE_STATUS_UNKNOWN",
        "total_bytes": "12345",
        "total_bytes_indexed": "12345",
        "total_file_count": "12345"
      },
      "last_indexing_job": {
        "completed_datasources": 123,
        "created_at": "2023-01-01T00:00:00Z",
        "data_source_uuids": [
          "example string"
        ],
        "finished_at": "2023-01-01T00:00:00Z",
        "knowledge_base_uuid": "123e4567-e89b-12d3-a456-426614174000",
        "phase": "BATCH_JOB_PHASE_UNKNOWN",
        "started_at": "2023-01-01T00:00:00Z",
        "status": "INDEX_JOB_STATUS_UNKNOWN",
        "tokens": 123,
        "total_datasources": 123,
        "total_items_failed": "12345",
        "total_items_indexed": "12345",
        "total_items_skipped": "12345",
        "updated_at": "2023-01-01T00:00:00Z",
        "uuid": "123e4567-e89b-12d3-a456-426614174000"
      },
      "region": "example string",
      "spaces_data_source": {
        "bucket_name": "example name",
        "item_path": "example string",
        "region": "example string"
      },
      "updated_at": "2023-01-01T00:00:00Z",
      "uuid": "123e4567-e89b-12d3-a456-426614174000",
      "web_crawler_data_source": {
        "base_url": "example string",
        "crawling_option": "UNKNOWN",
        "embed_media": true
      }
    }
  ],
  "links": {
    "pages": {
      "first": "example string",
      "last": "example string",
      "next": "example string",
      "previous": "example string"
    }
  },
  "meta": {
    "page": 123,
    "pages": 123,
    "total": 123
  }
}

Returns Examples

200 example

{
  "knowledge_base_data_sources": [
    {
      "aws_data_source": {
        "bucket_name": "example name",
        "item_path": "example string",
        "region": "example string"
      },
      "bucket_name": "example name",
      "created_at": "2023-01-01T00:00:00Z",
      "dropbox_data_source": {
        "folder": "example string"
      },
      "file_upload_data_source": {
        "original_file_name": "example name",
        "size_in_bytes": "12345",
        "stored_object_key": "example string"
      },
      "item_path": "example string",
      "last_datasource_indexing_job": {
        "completed_at": "2023-01-01T00:00:00Z",
        "data_source_uuid": "123e4567-e89b-12d3-a456-426614174000",
        "error_details": "example string",
        "error_msg": "example string",
        "failed_item_count": "12345",
        "indexed_file_count": "12345",
        "indexed_item_count": "12345",
        "removed_item_count": "12345",
        "skipped_item_count": "12345",
        "started_at": "2023-01-01T00:00:00Z",
        "status": "DATA_SOURCE_STATUS_UNKNOWN",
        "total_bytes": "12345",
        "total_bytes_indexed": "12345",
        "total_file_count": "12345"
      },
      "last_indexing_job": {
        "completed_datasources": 123,
        "created_at": "2023-01-01T00:00:00Z",
        "data_source_uuids": [
          "example string"
        ],
        "finished_at": "2023-01-01T00:00:00Z",
        "knowledge_base_uuid": "123e4567-e89b-12d3-a456-426614174000",
        "phase": "BATCH_JOB_PHASE_UNKNOWN",
        "started_at": "2023-01-01T00:00:00Z",
        "status": "INDEX_JOB_STATUS_UNKNOWN",
        "tokens": 123,
        "total_datasources": 123,
        "total_items_failed": "12345",
        "total_items_indexed": "12345",
        "total_items_skipped": "12345",
        "updated_at": "2023-01-01T00:00:00Z",
        "uuid": "123e4567-e89b-12d3-a456-426614174000"
      },
      "region": "example string",
      "spaces_data_source": {
        "bucket_name": "example name",
        "item_path": "example string",
        "region": "example string"
      },
      "updated_at": "2023-01-01T00:00:00Z",
      "uuid": "123e4567-e89b-12d3-a456-426614174000",
      "web_crawler_data_source": {
        "base_url": "example string",
        "crawling_option": "UNKNOWN",
        "embed_media": true
      }
    }
  ],
  "links": {
    "pages": {
      "first": "example string",
      "last": "example string",
      "next": "example string",
      "previous": "example string"
    }
  },
  "meta": {
    "page": 123,
    "pages": 123,
    "total": 123
  }
}