Lucene and DB Query Limits

A poorly designed query sent to Lucene, or the PosgreSQL database index, can return an extremely large number of objects, which are then resolved. This can require a great deal of Java heap memory, take a long time to process, and the results of a single query may never be actually returned to the user's browser interface. Also, the processing of so much data can launch the garbage collection process in a loop. Garbage collection ties up the CPU, freezes the Java virtual machine for up to several minutes, and can lead to an endless loop and an Out of Memory (OOM) error.

Administrators can set the following two system configuration properties to limit the size of query results. One causes very large requests initiated by users from the web interface to be rejected and stopped from running. Consequently, users are forced to optimize queries to reduce the number of returned objects to less than the configured limits. Another detects large requested initiated by other means and writes information to log files.

```
com.siemens.polarion.platform.hardSearchSizeLimit
```
Defines a hard limit of maximum Lucene and database query result size to prevent Out of Memory Errors. If this query limit is exceeded, the warning message below appears or an exception is thrown to API/WS calls and the query is stopped.
Example:
We are sorry, but your query below returns more than the maximum allowed limit of 100 000 objects. Please refine it or refer to the Lucene and DB Query Limits section in Polarion Help.
```
[query that caused the warning]
```
```
com.siemens.polarion.platform.softSearchSizeLimit
```
Defines a soft limit for the maximum Lucene and database query result size to prevent Out of Memory Errors. If exceeded, a warning message is written to the log files, but the query is not stopped.
Warning:
If you set too low a value in this property, and many users subsequently run queries that exceed the limit, then the logs can become heavily polluted.

Note:

After a period of evaluation of internal and customer-supplied usage data, it is possible that the default value of the above properties might change in a future release, or the possibility to configure these limits may be removed, and the limit values hard-coded. If so, it will be announced in the Configuration.txt file bundled with the relevant release distributions, and documented in online Help.

For current default values and other information, see the System Configuration Properties Reference document in the Polarion section of Siemens' Doc Center for more information.

Tips for administrators

The above properties serve as a rearguard against overly large query results. The best defense is prevention ― users trained to write optimized queries that do not stress the system unduly.
Administrators may want to monitor the log for instances of the soft limit being invoked. If present too often, it's indicative of the need to train users to construct better optimized queries.
Administrators should avoid increasing the default values of the above properties. Such an increase is not recommended, as it increases the danger of OOM errors. Again, user education, and the use of optimized queries are the recommended best practices.