Good morning,
we have new xRes development database servers
db01.dev.xres.gs.xtrav.de <http://dev.xres.gs.xtrav.de/> and slave
db02.dev.xres.gs.xtrav.de
Most databases are copied from test03.dev.xres, „xres_dev_contingentflights“ is copied from devil-b, xibe and xibe_dev has already been there.
Login and permissions:
Benutzer "deployment"
Schema Verwaltung: Anlegen/Entfernen von Datenbanken, Daten-Administration
Gemeinsames Passwort für alle Kunden, keine Source-Adressen Einschränkung
Login für DBA Team und bamboo01 deployments
<>Benutzer "xres_<customer>"
eingeschränkter Zugriff aus xRes/xAdmin heraus
extra-Passwort pro <customer>
Login für Entwickler (Heidi SQL) und xRes Anwendung
<>Benutzer "xres"
MySQL DEFINER Benutzer für den Betrieb von Triggern und Procedures.
Kein Login für jemanden, Passwort gescrambled
Berechtigungen wie "xres_<customer>"
Login von überall ('%' - wichtig für Hotel Importe)
DEPLOYMENT_USER='deployment'
DEPLOYMENT_PASS='dZwysnPFmNUxYB0PKm'
CUSTOMER_USER='xres_dev'
CUSTOMER_DB='xres_dev'
CUSTOMER_PASS='Ero7Auwah6Ueghi1goodahng9aeXuqu7'
DEFINE_USER='xres'
DEFINE_PASS=<scrambled>
Please tell me when you discover any troubles logging in.
The database server devil-b and it’s companion devil-c have been put to EOLife and will be taken down early January. Data written to it from now will be lost then.
Kind Regards
Gunnar
--
Mit freundlichen Grüßen
Gunnar Mann
- Systemadministration -
________________________________________________________
TraSo GmbH
Nonnenstraße 42
D-04229 Leipzig
Tel.: +49 341 355 740 76
Fax: +49 341 355 740 21
E-Mail: g.mann(a)traso.de <mailto:g.mann@traso.de>
<https://www.traso.de/> <https://www.facebook.com/TraSoGmbH> <https://www.xing.com/companies/trasogmbh>
________________________________________________________
Geschäftsführer: Haiko Gerdes
Handelsregister: Amtsgericht Leipzig, HRB 21850 <https://www.kununu.com/de/traso1>
________________________________________________________
Geschäftsführer: Haiko Gerdes
Handelsregister: Amtsgericht Leipzig, HRB 21850
<https://www.kununu.com/de/traso1>
Dear all,
we experienced a severe performance impact in a php application writing small data chunks to our GlusterFS network file system. Some details can be found here <https://aimeos.org/tips/modern-php-applications-up-to-110x-slower-on-networ…> and there <https://serverfault.com/questions/86000/can-you-explain-why-i-shouldnt-serv…> - the baseline is that file_exists, is_dir and all functions using system call lstat can not be cached in APC and cause GlusterFS to check with the quorum of servers during the operation.
Python and native linux applications ( C ) do not have this problem, so there is no obvious need to fix at system side. The problem did not arise on xRes data importing where only single big writes are placed on GlusterFS. Testing around with php realpath_cache_size and opcache settings did not help.
Jacob implemented a simple memory buffering for collecting file writes into bigger chunks which turned out to be the single most helpful workaround. Furthermore it helps to not use file_put_contents inside big loops but open and close files only once outside a loop. See examples:
Best: buffering and fopen()/fclose() called only once, 0.8 sec:
$line = str_repeat('x', 128) . PHP_EOL;
$buffer = '';
$fh = fopen('/data/php_test.txt', 'w+');
for ($i = 0; $i < 100000; $i++) {
$buffer .= $line;
if ($i % 1000 === 0) {
fwrite($fh, $buffer);
$buffer = '';
}
}
fwrite($fh, $buffer);
fclose($fh);
2.5 times slower: buffering and open/write/close inside loop, 2.0 sec:
$line = str_repeat('x', 128) . PHP_EOL;
$buffer = '';
for ($i = 0; $i < 100000; $i++) {
$buffer .= $line;
if ($i % 1000 === 0) {
file_put_contents('/data/php_test.txt', $buffer, FILE_APPEND);
$buffer = '';
}
}
file_put_contents('/data/php_test.txt', $buffer, FILE_APPEND);
Worst: unbuffered plain open/write/close inside loop: 186 sec
$line = str_repeat('x', 128) . PHP_EOL;
for ($i = 0; $i < 100000; $i++) {
file_put_contents('/data/php_test.txt', $line, FILE_APPEND);
}
Having fopen(), fclose() outside of loops may help writing to hard disks, too.
Any more ideas?
--
Mit freundlichen Grüßen
Gunnar Mann
- Systemadministration -
________________________________________________________
TraSo GmbH
Nonnenstraße 42
D-04229 Leipzig
Tel.: +49 341 355 740 76
Fax: +49 341 355 740 21
E-Mail: g.mann(a)traso.de <mailto:g.mann@traso.de>
<https://www.traso.de/> <https://www.facebook.com/TraSoGmbH> <https://www.xing.com/companies/trasogmbh>
________________________________________________________
Geschäftsführer: Haiko Gerdes
Handelsregister: Amtsgericht Leipzig, HRB 21850 <https://www.kununu.com/de/traso1>
________________________________________________________
Geschäftsführer: Haiko Gerdes
Handelsregister: Amtsgericht Leipzig, HRB 21850
<https://www.kununu.com/de/traso1>
