technical development goals for xRes in 2019
Moin xRes developers, First of all THANK YOU for taking part and supporting the process of technical development of xRes and its infrastructure. I feel very glad to see You interested and active in this area. After several years this process of technical improvements was successful, but often handled unplanned and based on the engagement of single developers. But we reached (in example) the following aims: * using HHVM and PHP 7 in production * running xRes on CentOS * setting up a new xRes installation within only a few days * scaling service, data and cache machines to almost unlimited ressources * refacturing a huge part of xRes code * adding (since January 2014) UnitTesting to xRes: Tests: 6445, Assertions: 25715 * and MUCH more ... For the year 2019 we did today talk about the following topics to be done / solved / planned: 1. We will finish to move old service01/02 Debian systems to new _*CentOS*_ machines. -> default PHP version will be 5.6 -> _*Bamboo*_ build process MUST be migrated to PHP 5.6 or 7.1 (or both) BOFORE to run UnitTests with new MongoDB driver -> _*MongoDB diver*_ must be replaced under PHP 5.6 within an xRes release (it is not compatible to PHP 7) -> after that default _*PHP*_ version can be switched to _*7.1*_ -> all xRes customers (except LMX) will move to _*ServiceXY.BC*_ booking clusters ... only LMX gets dedicated ressources 2. Currently _*ZendFramework 1 does NOT support PHP 7.2*_. We'll need a solution for this problem considering PHP 7.1 is supported until end of 2019. 3. Is it helpful to _*split xRes git repository*_ into separate repos? We'll have a developer meeting about this in January 2019. 4. Rene mentions existing problems in the _*structure of booking in xRes database*_. In a "Mittwochs-Session" he will present these problems and we'll discuss possible solutions and the advantages / disadvantages and costs. 5. Currently _*logging and monitoring*_ are a mess in xRes code and in TraSo infrastructure. We will have another "Mittwochs-Session" prepared by Viktoras to discuss about: -> How can developers use NEW code structures / adapters / interfaces and help cleaning up inhomogeneous logging structure in xRes. -> How can and should a future logging infrastructure look like? What technical requirements do we know about a future logging solution? -> Which requirements do xRes customers and TraSo management have on monitoring for technical and business monitoring of xRes systems? 6. _*Dependency Injection Container*_ => Georgii will prepare a "Mittwochs Session" to show xRes developers the usage of a DIC while refactoring existing code in a live demonstration. 7. The current _*xRes database deployment system*_ based on phing/dbdeploy has serveral disadvantages. Jacob will present pingx as a possible solution for these problems. I will ask Felix to add his experience writing an own db deployment for xMid 1.5 years ago. 8. For several reasons we could need another system to run non-webserver jobs we currently start as "CRON". Requirements for such a _*new CronJob system*_ are i.e. dependency jobs (start job B when job A is ready), a more than minutely running job (i.e. all 30 seconds or more often) and a GUI based scheduling (start a job on click, see job status, kill a job, ...). 9. Starting with new xRes customers SIT, RZP and SAR we are accidentially _*using MariaDB*_ from CentOS repository as xRes database ... and we will soon see if this works or if we broke a hell loose. But to be "in time" with current versions of MySQL we should be abled to use -> MySQL 5.6 / 5.7 or 8.0, -> MariaDB 10.2 or 10.3 or -> PerconaDB 5.6 In case of PerconaDB we SHOULD take care about the frequently appearing problem of _*DB fragmentation*_ we see i.e. in CacheDb systems. 10. The whole xRes system was setup and is still running in _*ISO-8859 Latin-1 encoding*_. Again and again we are having problems converting this stuff to UTF8 or from external UTF8 to Latin-1 and backwards. Our customers submit MANY tickets in context of encoding problems, wrong displayed informations or booking problems related to destroyed traveller names. We SHOULD finally solve this problem. In addition xRes is on some places of xAdmin abled to be used multilingual ... but this functionality does currently not cover all xAdmin pages and is not implemented in xRes booking core at all. 11. Since the feature of DSGVO+PRRL in summer 2018 we use Redis as a caching system for xRes booking data. On some xRes installations a separate Redis service is used, customers running on service booking cluster already use a shared Redis cluster (because of a newer driver in higher PHP version). After point 1 on top of this mail we should monitor the state of this Redis infrastructure and bring ALL xRes installations to run on the central _*Redis cluster*_. 12. Since decades an xRes booking has ONE status for the whole booking. Neither any selling plattform nor a midoffice system handles bookings this way ... all of them require a status per service (i.e. per hotel, flight or transfer). We are getting more and more problems matching this global status per booking from and to service status ... and we finally should introduce _*a status for every single service in a booking*_. 13. The hotel providers GTA and TOUR have been bought by another hotel provider company Hotelbeds HBD. HBD communicated, the offers currently imported and booked via GTA and TOUR gateways will one day be available via HBD booking gateway only. I am very looking forward to _*deactivation of the GTA import servers*_ we had to setup to manage the import process GTA provides. Please now: * Give me (to the developers list) feedback about these topics. How do YOU prioritize the topics? Which topics can YOU help to solve? * Add one, two, many of the important, urgent or just interesting topics we have NOT been abled to talk about in todays meeting. BG SRK -- Mit freundlichen Grüßen Stefan Rank-Kunitz - Technical Director - ________________________________________________________ TraSo GmbH Nonnenstraße 42 D-04229 Leipzig Tel.: +49 341 355 740 - 43 E-Mail: s.rank-kunitz@traso.de ________________________________________________________ Handelsregister: Amtsgericht Leipzig, HRB 21850
Hello Stefan, thank you very much for organization of the meeting and the email. Now, that we have listed the problems that we want to solve we can approach them more easily. I would like to suggest a plan for the first two points, a roadmap of actions we need to do, in order to achieve these goals. 1. Migration to CentOS and PHP 7. Thanks to everyone and especially Sören, we have a good progress in this project. The next step would be to move our customers' service machines into BC one after another. I have already suggested, that Denny creates a new instance in the BC for 4YBK. This way, step by step we will solve this as well. However, as the only admin Denny has A LOT to do, so this may not be as quick. I really hope we can finish transition until mid January-February, before the "hot season" begins. 2. Zend Framework transition. As mentioned, we are slowed down by ZF1 here. This transition will be a loooooong one, so I believe we should start as soon as possible, before we find ourselves on a not supported PHP version. There is one difference between ZF1 and modern frameworks. Modern frameworks do not try to give you an environment to build your project into, but rather provide a set of libraries, which you can use separately. I want to suggest, that we also move this direction not to be dependent on one single huge framework, but rather replace it's parts with newer libraries. The easiest and most convenient way to it is from my point of view to include new ZF (2/3?) libraries into our composer.json: Zend\Db, Zend\Http, Zend\Filter, Zend\View, Zend\Validator. With Zend libraries it should be much easier to update as with any other. This way, we can replace the usage of old ZF1 parts with the newer ZF2 parts and later probably with something else if we want. We will not be bound to a single framework anymore and we will not have to change that many parts at the same time! Other topics will be discussed further in the future and then we can make a plan for them as well. Best regards Viktoras Bezaras On 12.12.2018 17:23, Stefan Rank-Kunitz wrote:
Moin xRes developers,
First of all THANK YOU for taking part and supporting the process of technical development of xRes and its infrastructure. I feel very glad to see You interested and active in this area. After several years this process of technical improvements was successful, but often handled unplanned and based on the engagement of single developers. But we reached (in example) the following aims:
* using HHVM and PHP 7 in production * running xRes on CentOS * setting up a new xRes installation within only a few days * scaling service, data and cache machines to almost unlimited ressources * refacturing a huge part of xRes code * adding (since January 2014) UnitTesting to xRes: Tests: 6445, Assertions: 25715 * and MUCH more ...
For the year 2019 we did today talk about the following topics to be done / solved / planned:
1. We will finish to move old service01/02 Debian systems to new _*CentOS*_ machines. -> default PHP version will be 5.6 -> _*Bamboo*_ build process MUST be migrated to PHP 5.6 or 7.1 (or both) BOFORE to run UnitTests with new MongoDB driver -> _*MongoDB diver*_ must be replaced under PHP 5.6 within an xRes release (it is not compatible to PHP 7) -> after that default _*PHP*_ version can be switched to _*7.1*_ -> all xRes customers (except LMX) will move to _*ServiceXY.BC*_ booking clusters ... only LMX gets dedicated ressources
2. Currently _*ZendFramework 1 does NOT support PHP 7.2*_. We'll need a solution for this problem considering PHP 7.1 is supported until end of 2019.
3. Is it helpful to _*split xRes git repository*_ into separate repos? We'll have a developer meeting about this in January 2019.
4. Rene mentions existing problems in the _*structure of booking in xRes database*_. In a "Mittwochs-Session" he will present these problems and we'll discuss possible solutions and the advantages / disadvantages and costs.
5. Currently _*logging and monitoring*_ are a mess in xRes code and in TraSo infrastructure. We will have another "Mittwochs-Session" prepared by Viktoras to discuss about: -> How can developers use NEW code structures / adapters / interfaces and help cleaning up inhomogeneous logging structure in xRes. -> How can and should a future logging infrastructure look like? What technical requirements do we know about a future logging solution? -> Which requirements do xRes customers and TraSo management have on monitoring for technical and business monitoring of xRes systems?
6. _*Dependency Injection Container*_ => Georgii will prepare a "Mittwochs Session" to show xRes developers the usage of a DIC while refactoring existing code in a live demonstration.
7. The current _*xRes database deployment system*_ based on phing/dbdeploy has serveral disadvantages. Jacob will present pingx as a possible solution for these problems. I will ask Felix to add his experience writing an own db deployment for xMid 1.5 years ago.
8. For several reasons we could need another system to run non-webserver jobs we currently start as "CRON". Requirements for such a _*new CronJob system*_ are i.e. dependency jobs (start job B when job A is ready), a more than minutely running job (i.e. all 30 seconds or more often) and a GUI based scheduling (start a job on click, see job status, kill a job, ...).
9. Starting with new xRes customers SIT, RZP and SAR we are accidentially _*using MariaDB*_ from CentOS repository as xRes database ... and we will soon see if this works or if we broke a hell loose. But to be "in time" with current versions of MySQL we should be abled to use -> MySQL 5.6 / 5.7 or 8.0, -> MariaDB 10.2 or 10.3 or -> PerconaDB 5.6 In case of PerconaDB we SHOULD take care about the frequently appearing problem of _*DB fragmentation*_ we see i.e. in CacheDb systems.
10. The whole xRes system was setup and is still running in _*ISO-8859 Latin-1 encoding*_. Again and again we are having problems converting this stuff to UTF8 or from external UTF8 to Latin-1 and backwards. Our customers submit MANY tickets in context of encoding problems, wrong displayed informations or booking problems related to destroyed traveller names. We SHOULD finally solve this problem. In addition xRes is on some places of xAdmin abled to be used multilingual ... but this functionality does currently not cover all xAdmin pages and is not implemented in xRes booking core at all.
11. Since the feature of DSGVO+PRRL in summer 2018 we use Redis as a caching system for xRes booking data. On some xRes installations a separate Redis service is used, customers running on service booking cluster already use a shared Redis cluster (because of a newer driver in higher PHP version). After point 1 on top of this mail we should monitor the state of this Redis infrastructure and bring ALL xRes installations to run on the central _*Redis cluster*_.
12. Since decades an xRes booking has ONE status for the whole booking. Neither any selling plattform nor a midoffice system handles bookings this way ... all of them require a status per service (i.e. per hotel, flight or transfer). We are getting more and more problems matching this global status per booking from and to service status ... and we finally should introduce _*a status for every single service in a booking*_.
13. The hotel providers GTA and TOUR have been bought by another hotel provider company Hotelbeds HBD. HBD communicated, the offers currently imported and booked via GTA and TOUR gateways will one day be available via HBD booking gateway only. I am very looking forward to _*deactivation of the GTA import servers*_ we had to setup to manage the import process GTA provides.
Please now:
* Give me (to the developers list) feedback about these topics. How do YOU prioritize the topics? Which topics can YOU help to solve? * Add one, two, many of the important, urgent or just interesting topics we have NOT been abled to talk about in todays meeting.
BG SRK
-- Mit freundlichen Grüßen
Stefan Rank-Kunitz - Technical Director - ________________________________________________________ TraSo GmbH
Nonnenstraße 42 D-04229 Leipzig Tel.: +49 341 355 740 - 43
E-Mail: s.rank-kunitz@traso.de ________________________________________________________ Handelsregister: Amtsgericht Leipzig, HRB 21850
_______________________________________________ Entwicklung mailing list Entwicklung@lists.traso.de https://lists.traso.de/listinfo/entwicklung
-- Best regards Viktoras Bezaras - Integrations Leader - ________________________________________________________ TraSo GmbH Nonnenstraße 42 D-04229 Leipzig Tel.: +49 341 355 740 44 E-Mail: v.bezaras@traso.de Internet: http://www.traso.de ________________________________________________________ Geschäftsführer: Haiko Gerdes Handelsregister: Amtsgericht Leipzig, HRB 21850
Here is my priority for the known points plus additional topics from my site. 9. priority is given by errors ;) - multi customer. Currently there are some processes that are not able to run in a multi customer setup. For example the using of the archive mount or /var/xres/import-export location. 1. 7. 10. 11. 2. 5. 4 and 12 - Refactoring hotel database structure - C6000 disaster. Our admins plan a diskless using of our C6000 hardware, to run processes that do not required hard disk. 3. 6. 8. 13. - Refactor the multi operator system. From the beginning there was no plan for a multi operator system. The current implementation is a patchwork based on customer requirements. We need a exact delimitation and a better concept. Perhaps the xmid implementation helps us. I think that's enough for 2019 ;) Thanks for the teamwork. Am 12.12.2018 um 17:23 schrieb Stefan Rank-Kunitz:
Moin xRes developers,
First of all THANK YOU for taking part and supporting the process of technical development of xRes and its infrastructure. I feel very glad to see You interested and active in this area. After several years this process of technical improvements was successful, but often handled unplanned and based on the engagement of single developers. But we reached (in example) the following aims:
* using HHVM and PHP 7 in production * running xRes on CentOS * setting up a new xRes installation within only a few days * scaling service, data and cache machines to almost unlimited ressources * refacturing a huge part of xRes code * adding (since January 2014) UnitTesting to xRes: Tests: 6445, Assertions: 25715 * and MUCH more ...
For the year 2019 we did today talk about the following topics to be done / solved / planned:
1. We will finish to move old service01/02 Debian systems to new _*CentOS*_ machines. -> default PHP version will be 5.6 -> _*Bamboo*_ build process MUST be migrated to PHP 5.6 or 7.1 (or both) BOFORE to run UnitTests with new MongoDB driver -> _*MongoDB diver*_ must be replaced under PHP 5.6 within an xRes release (it is not compatible to PHP 7) -> after that default _*PHP*_ version can be switched to _*7.1*_ -> all xRes customers (except LMX) will move to _*ServiceXY.BC*_ booking clusters ... only LMX gets dedicated ressources
2. Currently _*ZendFramework 1 does NOT support PHP 7.2*_. We'll need a solution for this problem considering PHP 7.1 is supported until end of 2019.
3. Is it helpful to _*split xRes git repository*_ into separate repos? We'll have a developer meeting about this in January 2019.
4. Rene mentions existing problems in the _*structure of booking in xRes database*_. In a "Mittwochs-Session" he will present these problems and we'll discuss possible solutions and the advantages / disadvantages and costs.
5. Currently _*logging and monitoring*_ are a mess in xRes code and in TraSo infrastructure. We will have another "Mittwochs-Session" prepared by Viktoras to discuss about: -> How can developers use NEW code structures / adapters / interfaces and help cleaning up inhomogeneous logging structure in xRes. -> How can and should a future logging infrastructure look like? What technical requirements do we know about a future logging solution? -> Which requirements do xRes customers and TraSo management have on monitoring for technical and business monitoring of xRes systems?
6. _*Dependency Injection Container*_ => Georgii will prepare a "Mittwochs Session" to show xRes developers the usage of a DIC while refactoring existing code in a live demonstration.
7. The current _*xRes database deployment system*_ based on phing/dbdeploy has serveral disadvantages. Jacob will present pingx as a possible solution for these problems. I will ask Felix to add his experience writing an own db deployment for xMid 1.5 years ago.
8. For several reasons we could need another system to run non-webserver jobs we currently start as "CRON". Requirements for such a _*new CronJob system*_ are i.e. dependency jobs (start job B when job A is ready), a more than minutely running job (i.e. all 30 seconds or more often) and a GUI based scheduling (start a job on click, see job status, kill a job, ...).
9. Starting with new xRes customers SIT, RZP and SAR we are accidentially _*using MariaDB*_ from CentOS repository as xRes database ... and we will soon see if this works or if we broke a hell loose. But to be "in time" with current versions of MySQL we should be abled to use -> MySQL 5.6 / 5.7 or 8.0, -> MariaDB 10.2 or 10.3 or -> PerconaDB 5.6 In case of PerconaDB we SHOULD take care about the frequently appearing problem of _*DB fragmentation*_ we see i.e. in CacheDb systems.
10. The whole xRes system was setup and is still running in _*ISO-8859 Latin-1 encoding*_. Again and again we are having problems converting this stuff to UTF8 or from external UTF8 to Latin-1 and backwards. Our customers submit MANY tickets in context of encoding problems, wrong displayed informations or booking problems related to destroyed traveller names. We SHOULD finally solve this problem. In addition xRes is on some places of xAdmin abled to be used multilingual ... but this functionality does currently not cover all xAdmin pages and is not implemented in xRes booking core at all.
11. Since the feature of DSGVO+PRRL in summer 2018 we use Redis as a caching system for xRes booking data. On some xRes installations a separate Redis service is used, customers running on service booking cluster already use a shared Redis cluster (because of a newer driver in higher PHP version). After point 1 on top of this mail we should monitor the state of this Redis infrastructure and bring ALL xRes installations to run on the central _*Redis cluster*_.
12. Since decades an xRes booking has ONE status for the whole booking. Neither any selling plattform nor a midoffice system handles bookings this way ... all of them require a status per service (i.e. per hotel, flight or transfer). We are getting more and more problems matching this global status per booking from and to service status ... and we finally should introduce _*a status for every single service in a booking*_.
13. The hotel providers GTA and TOUR have been bought by another hotel provider company Hotelbeds HBD. HBD communicated, the offers currently imported and booked via GTA and TOUR gateways will one day be available via HBD booking gateway only. I am very looking forward to _*deactivation of the GTA import servers*_ we had to setup to manage the import process GTA provides.
Please now:
* Give me (to the developers list) feedback about these topics. How do YOU prioritize the topics? Which topics can YOU help to solve? * Add one, two, many of the important, urgent or just interesting topics we have NOT been abled to talk about in todays meeting.
BG SRK
-- Mit freundlichen Grüßen
Stefan Rank-Kunitz - Technical Director - ________________________________________________________ TraSo GmbH
Nonnenstraße 42 D-04229 Leipzig Tel.: +49 341 355 740 - 43
E-Mail: s.rank-kunitz@traso.de ________________________________________________________ Handelsregister: Amtsgericht Leipzig, HRB 21850
_______________________________________________ Entwicklung mailing list Entwicklung@lists.traso.de https://lists.traso.de/listinfo/entwicklung
-- -- Sören Pestner - Entwickler - TraSo GmbH Nonnenstraße 42 D-04229 Leipzig telefon: +49 341 355 740 - 49 email: s.pestner@traso.de Geschäftsführer: Haiko Gerdes Handelsregister: Amtsgericht Leipzig (HRB 21850)
participants (3)
-
Soeren Pestner -
Stefan Rank-Kunitz -
Viktoras Bezaras