Extracting data into memory

The first indexing process has a threshold for restricting the amount of memory the extracted data buffer can consume before the data is written to disk. The greater the memory allocation, the more efficient the entire indexing process is, because more data can be handled simultaneously.

Table 3-42: General upload parameters

Parameter

Default

Description

omniq.index.buffer.maxMemory

10MB

Indexing is more efficient if many documents are indexed in a batch. The buffer's maximum memory allocation determines how many documents are processed in each batch.

omniq.indexer.maxDocumentSize

10MB

Sets the maximum document size to be indexed.

NoteVery long documents have an adverse effect on the query results.