Eric Adams - Director of Operations
A recent upgrade of our underlying infrastructure caused a configuration to not be applied properly, which in turn caused ingestions to process much slower than normal. A fix was applied late Monday night around 23:30 and it was monitored throughout the night with success.
All ingestions were slower to process than normal from Sunday night and through Monday.
The upgrade of the underlying infrastructure caused an environment variable that determines how Kimono processes
MatchKeys for ingestions to no longer be applied.
Kimono applied a hotfix release with a new environment variable to apply the new configuration in the new infrastructure.
|Monday, October 8th, 06:00||It was identified that many collections that had started the previous night were not processing at the normal rate. The Kimono Engin|
|Monday, October 8th, 09:00||Processing issues were escalated from the Operations team to the Engineering team to begin working on a resolution|
|Monday, October 8th, 18:00||Kimono Engineering identified the potential issue and began working on a hotfix to remediate the slowness.|
|Monday, October 8th, 22:00||Hotfix was deployed and tested in our QA environment and proven to have the necessary changes.|
|Monday, October 8th, 23:15||The hotfix was successfully deployed to the Production environment.|