beenull/orcinus-search
1
0
Fork 0
Code Issues Pull requests Actions Packages Projects Releases Wiki Activity

Update search.php

Browse source 
Don't allow JSON requests to trigger a new crawl or end a stuck one, since the requests may come too fast to handle.

Only allow triggering a new crawl if more than 'sp_timeout_crawl' seconds have passed since we canceled the previous one.

In the future, we might give each successfully initiated crawl a unique ID and then only allow sending a failure email once if it has failed. A very busy search engine is probably indistinguishable from a rapid-fire series of JSON requests.
This commit is contained in:
Brian Huisman 2023-12-06 10:14:18 -05:00
parent a125060d7f
commit 5d990e44b0
1 changed files with 10 additions and 8 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

18
orcinus/search.php
View file

@ -789,6 +789,12 @@ if ($_RDATA['s_searchable_pages']) {
} // No searchable pages in search database
// Output JSON and exit if requested
if (isset($_REQUEST['json'])) {
header('Content-type: application/json; charset='.$_ODATA['s_charset']);
die(json_encode($_SDATA['json'], JSON_INVALID_UTF8_IGNORE));
}
// If the crawler 'time_start' is more than 'timeout_crawl'
// seconds ago, the crawler is probably stuck. Unstick it.
@ -813,7 +819,8 @@ if (OS_getValue('sp_crawling') && time() - $_ODATA['sp_time_start'] > $_ODATA['s
// ***** Trigger another crawl
if ($_ODATA['sp_interval'] && !$_ODATA['sp_crawling'] &&
time() - $_ODATA['sp_time_end_success'] > $_ODATA['sp_interval'] * 3600) {
time() - $_ODATA['sp_time_end_success'] > $_ODATA['sp_interval'] * 3600 &&
$_ODATA['sp_time_end'] - $_ODATA['sp_timeout_crawl'] < time()) {
// If we can only trigger the crawl during certain time period
if ($_ODATA['sp_interval_start'] != $_ODATA['sp_interval_stop']) {
@ -876,10 +883,5 @@ if ($_ODATA['sp_interval'] && !$_ODATA['sp_crawling'] &&
}
}
// Output JSON and exit if requested
if (isset($_REQUEST['json'])) {
header('Content-type: application/json; charset='.$_ODATA['s_charset']);
die(json_encode($_SDATA['json'], JSON_INVALID_UTF8_IGNORE));
// Just log this query and do not post results
} else if (isset($_REQUEST['log'])) die(); ?>
// If 'log' argument is set, just log this query and do not post results
if (isset($_REQUEST['log'])) die(); ?>