Hey, we are currently ingesting Piwik’s raw sessions/events on a daily basis to our data warehouse.
We wanted to adapt it to our other systems and ingest it 4 times a day. Our approach would need to call the API to grab data after a specific timestamp (last extraction) until now.
Is this something possible?
The timestamp in the query is precise down to a day, so I’d suggest to try using the offset parameter to achieve that. Then with each run you have to store the count of the records downloaded and use the sum of records from previous downloads as offset for the next one. I assume those should not overlap if you use “order_by” on session_id.
Thanks! We will try this out and see how that goes.