Podium - Contact CSV Uploads may not be uploading – Incident details

All systems operational

Contact CSV Uploads may not be uploading

Resolved
Major outage
Started about 9 hours agoLasted about 9 hours

Affected

Contacts

Degraded performance from 4:45 PM to 1:53 AM

Updates
  • Resolved
    Resolved

    We've seen all accounts return to a stable speed. We apologize for how long this incident took, we'll be taking learning from these and implementing for better reliability in the future.

  • Update
    Update

    Most users should no longer be affected by this incident. We are seeing about 5-6% of our accounts still seeing some residue lag. We're still actively monitoring and attempting to remove this lag entirely

  • Monitoring
    Monitoring

    We implemented a fix and are currently monitoring the result. Verification of a resolution should be coming shortly.

  • Update
    Update

    We're still testing our hypothesis for new lead creation being delayed in Podium. We're getting closer to resolution but will provide an update as soon as our results start to show positive results

  • Identified
    Identified

    We believe we've identified the cause of the issue and are working to test our theory, hopefully resolving the issue. We'll continue to provide updates as we learn more.

  • Update
    Update

    We have discovered that contacts being created from all sources—CSV uploads, Integrations, and other Lead Sources are delayed. We're pulling in more engineering hands to help us resolve this incident. We'll provide an update soon.

  • Investigating
    Investigating
    We are investigating an issue where contact uploads through CSV Uploads are getting stuck in processing and not completing.