About Streams Lifetime
tldr; Streams should not go unbounded, find a limit in the domain or create one.
2016-08-25
Stream life
It took me several years to see the point, but I think it's crucial for your own sanity.
In Event Sourcing, people - me included - often say
1:
|
|
This is true, but it's only part of the story.
Greg told me how he was saving thousands of events per second, every second every day. So I was thinking saving ten to one hundrerd per second would be easy to manage.
But after a few month or a few years, things got difficult to maintain, and it could only get worse.
What I noticed, and was really implicit so I heard nobody talk about it before, is that Greg was working on stock market data. Stock exchange open typicaly from 9:30 a.m. to 4:00 p.m. ( NYSE ).
So at 4 PM, you can stop, aggregate the data, and start from consolidated data at 9:30 AM the next day, starting new streams.
It's quite different in contexts where there's no bell ringing...
The bell
Not all streams open in the morning and close at 4 PM.
You should still try to see wheter the underlying aggregate as a pre existing defined lifetime.
For instance, the price of an hotel room for as specific day in the future will not change anymore once that day has passed.
Some business processes have a limited lifetime and you should look for it and see whether the quantity of events that can happen in this lifetime is manageable. After the expiry date, the stream can be closed.
Unlimited streams.
Some streams seem unlimited in duration... But a stream containing events generated by a human will not exceed his life - max 120 years for now - and anyway not his usage of the system - a few years.
Humans don't do more that a few hundrend or thousands manual events a day which make such stream still limited in size.
This can still be a lot of events in the case of events generated from sensors. The same occures for events created by automated systems, or groups of humans etc...
In this case always come back to what accountants do !
Business years
Accountants use business years to consolidate the ledger entries. The ledger would grow unbounded without business years, which would prove difficult after a few years.
So each year, on a fixed date, accountants consolidate the ledger entries and start a new ledger starting with previous year consolidated result. They start a new event stream.
When an error is found in a previous ledger, it's not modified. A compensating entry is added to the current ledger.
For unbounded high volume streams, define business periods after which you consolidate the stream's events before starting a new stream. You can then archive old events for auditing.
Streams lifetime in FsUno
Streams in FsUno don't last more than one game, which is quite limited in time.
The typical number of events in a Uno Game is under 200 - this is just guess work, nothing scientfic.
This is far bellow the limit of unmanageability.
You should still be carefull about game automation. If some players create scrappers to automate games, you could end up with high speed long games that could potentialy exceed your expectations.
To avoid this, you could put a throtelling on game inputs to avoid high speed algorithms to compete.
Or accept it, and gradually split game streams after a given number of events, or limit game length in time depending on your business objectives.
