Archived status updates

All the events below have already been resolved and are here only for reference.
Visit the Platform status page to see the current situation.

Sun 19.11.2023

  Planned security update

Date range Sun 19.11.2023 - 22:20 to 22:30
Affected services Core features, Platform interfaces (API), Single Sign On (SSO), Terminals

Fri 03.11.2023

  Shifts from work schedules were incorrect in people's calendars after the update on Thursday 02.11.2023. This possibly also affected the calculated accrual values, making them incorrect. The problem is now solved.

Date range Thu 02.11.2023 13:30 - Fri 02.11.2023 13:00
Affected services Core features

Su 22.10.2023

  Planned security update

Date range Su 22.10.2023 - 22:05 to 22:25
Affected services Core features, Platform interfaces (API), Single Sign On (SSO), Terminals

Thr 12.10.2023

  'Support' link in the top-level navigation is not working

Date range 12.10.2023 - 19.10.2023
Affected services Link to support

• 12.10.2023 06:00 The link to the Nepton Support materials is not working.

Mon 9.10.2023

  Kiva work data problems

Date range 9.10.2023 07:00 - 12.10.2023 21:00
Affected services Trusted services

• 12.10.2023 21:00 - Kiva Work now works normally.
• 10.10.2023 17:00 - Based on the test results, we are doing additional checks.
• 9.10.2023 23:00 - A fix has been created and is being tested.
• 9.10.2023 12:00 - We are investigating problems with data usage in our new trusted service Kiva Work. Until resolved, Kiva Work reports may be inaccurate, and some users cannot access the trusted service.

Tue 10.10.2023

  Project search not working in Android devices

Date range 7.10.2023 07:00 - 11.10.11:10
Affected services Core features

• 11.10.2023 11:10 - Cause for the issue was identified and a fix for it was published
• 10.10.2023 16:00 - We are aware of a problem affecting users on Android mobile devices which prevents project searching. We are investigating.

Sat 7.10.2023

  Kiva Work is not available

Date range Sat 07.10.2023 - 05:30 to 08:30
Affected services Trusted services

• 7.10.2023 08:30 - The situation has been resolved. Users who were logged into Nepton must log in again.
• 7.10.2023 08:00 - We have identified a configuration issue preventing communication between Nepton and Kiva Work.
• 7.10.2023 05:30 - Kiva Work is not available. We are investigating the cause.

Sat 7.10.2023

  Planned service update to version 2.13.23

Date range Sat 07.10.2023 - 03:00 to 05:30
Affected services Core features, Trusted services

• More information on our release notes.

Thu 28.9.2023

  Some users are unable to record work hours using the Worktime > Work hours  > Current status, page

Date range 26.9.2023 13:00 - 28.9.2023 10:30
Affected services Work hour recording

• 28.9.2023 10:30 - The Current status page is working again after we released a minor fix.
• 26.9.2023 09:30 - We are investigating of a problem preventing some users from recording hours via the Worktime Work hours Current status, page.

Thu 21.9.2023

  Sign in with Google, was not working with the latest version of Chrome browser

Date range 21.9.2023 09:00 - 27.9.2023:13:14
Affected services Sign in with Google

• 27.9.2023:13:14 - Google sign-in works and solution to the problem has been released.
• 26.9.2023:16:50 - Solution to the problem will be released on the next patch release.
• 25.9.2023:16:15 - Investigation and fixing of the problem is still ongoing.
• 21.9.2023:12:15 - We are investigating a problem with the latest versions of Chrome browser, which have stopped working with our Google sign-in feature.

Su 17.09.2023

  Planned security update

Date range Su 17.09.2023 - 22:10 to 22:40
Affected services Core features, Platform interfaces (API), Single Sign On (SSO), Terminals

Tue 15.08.2023

  Service update took longer than expected

Date range Tue 15.08.2023 - 13:00 to 13:40
Affected services All features and services

Sun 13.08.2023

  Platform update

Date range Sun 13.08.2023 22:15 - 22:25
Affected services Core features, Platform interfaces (API), Single Sign On (SSO), Terminals

• Planned security update.

Sun 16.07.2023

  Platform update

Date range Sun 16.07.2023 22:00 - 22:20
Affected services Core features, Platform interfaces (API), Single Sign On (SSO), Terminals

• Planned security update.

.

Sat 08.07.2023

  Planned service update to version 2.13.0

Date range Sat 08.07.2023 - 03:00 to 05:30
Affected services Core features

• More information on our release notes.

Sun 18.06.2023

  Platform update

Date range Sun 18.06.2023 22:00 - 22:15
Affected services Core features, Platform interfaces (API), Single Sign On (SSO), Terminals

• Planned security update.

.

2.6.2023

  Updating the SSO certificate caused issues for some users.

Date range Fri 05.06.2023 18:00 - 19:00
Affected services: Single Sign On (SSO)

• 18:00 - Updating the SSO certificate has caused issues for some users. We are investigating the issue.
• 19:00 - Issue has been resolved and Single Sign On is working normally.

Sun 14.05.2023

  Platform update

Date range Sun 14.05.2023 22:30 - 22:45
Affected services Core features, Platform interfaces (API), Single Sign On (SSO), Terminals

• Planned security update.

Sat 13.05.2023

  Planned service update to version 2.12.0

Date range Sat 13.05.2023 - 03:00 to 05:30
Affected services All features and services

• More information on our release notes.

Tue 09.05.2023

  Service unavailable

Date range 09.05.2023 13:12 - 13:21
Affected services Core features, Platform interfaces (API), Scheduled integrations, Single Sign On (SSO), Terminals

• A planned update to our service on the maintenance window caused a longer service interruption than expected.

Sun 16.04.2023

  Platform update

Date range Sun 16.04.2023 22:00 - 22:15
Affected services Core features, Platform interfaces (API), Single Sign On (SSO), Terminals

• Planned security update.

Fri 24.03.2023

  Service degraded

Date range Fri 24.03.2023 07:47 - 08:20
Affected services All features and services

• 24.03.2023 07:47 - We have noticed slowness in the service and are now investigating the situation
• 24.03.2023 08:20 - We restarted parts of our service after investigations and the service is now operating normally

Sun 19.03.2023

  Platform update

Date range Sun 19.03.2023 22:00 - 22:25
Affected services Core features, Platform interfaces (API), Single Sign On (SSO), Terminals

• Planned security update.

Mon 13.03.2023

  Service degraded

Date range Mon 13.03.2023 12:00 - 15:58
Affected services Core features, Platform interfaces (API), Scheduled integrations, Single Sign On (SSO), Microsoft login

• 13.03.2023 15:10 - We have noticed slowness in the service and are now investigating the situation
• 13.03.2023 15:58 - We restarted parts of our service after investigations and the service is now operating normally

Sun 19.02.2023

  Platform update

Date range Sun 19.02.2023 22:30 - 22:45
Affected services Core features, Platform interfaces (API), Single Sign On (SSO), Terminals

• Planned security update.

Mon 13.2.2023

  Hotfix update

Date range 13.2.2023 4:30 - 4:45
Affected services Core features

• 2.11.19 hotfix update. Update can cause short momentary interruptions to the service.

Sun 15.01.2023

  Platform update

Date range Sun 15.01.2023 23:00 - 23:30
Affected services Core features, Platform interfaces (API), Single Sign On (SSO), Terminals

• Planned security update.

10.01.2023

  Service degraded

Date range Tue 10.01.2023 13:10 to 14:50
Affected services Core features, Platform interfaces (API), Scheduled integrations, Single Sign On (SSO), Terminals

• 10.01.2023 13:59 - We are aware of a problem in the service and are investigating it
• 10.01.2023 14:47 - A problem in service update is causing downtime periods in the service. We are restoring the service to its previous version to correct the situation.
• 10.01.2023 14:50 - Service is now operating normally.

31.12.2022

  Service degraded

Date range Sat 31.12.2022 18:15 to 19:40
Affects services: Worktime registration

• Service didn't allow time registrations

27.12.2022

  Planned service update to version 2.11.0

Date range Tue 27.12.2022 - 03:00 to 05:30
Affected services All features and services

• PLEASE NOTE: This update is made on an exceptional day (Tuesday), with less notice than usual. It is done to ensure service performance and to release significant work shift planning improvements in time for the new year.
• More information on our release notes.

21.12.2022

  RFID tag reading problems on some terminals

Date range Wed 21.12.2022 08:45
Affected services Terminals

• 21.12.2022 at 11:15 - All affected terminals have now been updated or are updating, and the issue is considered resolved.
• 21.12.2022 at 09:30 - Affected customers should restart their terminal to receive an update that corrects the RFID tag reading.
• 21.12.2022 at 08:45 - RFID tags are not working properly on some Nepton terminals. We have begun investigations. Where possible, please record work via go.nepton.com until the situation is resolved.

Sun 18.12.2022

  Platform update

Date range Sun 18.12.2022 22:00 - 22:20
Affected services Core features, Platform interfaces (API), Single Sign On (SSO), Terminals

• Planned security update.

Sun 20.11.2022

  Platform update

Date range Sun 20.11.2022 22:00 - 22:25
Affected services Core features, Platform interfaces (API), Single Sign On (SSO), Terminals

• Planned security update.

Mon 31.10.2022

  Currently our service is down. We are aware of the problem and have begun investigations.

Date range Tue 31.10.2022 13:14 -13:30
Affected services Core features

• Tue 31.10.2022 13:14 - We are aware of a problem impacting many parts of the platform and have begun investigations.
• Tue 31.10.2022 13:30 - Problems in the service have been resolved and the service is working normally. Outage in the service happened during normal maintenance window.

Thu 20.10.2022

  Exceptional downtime including some data loss

Date rangeThu 20.10.2022 15:26 - Tues 25.10.2022 06:50

Affected services All services

• Wed 9.11.2022 16:00 - Full description of the downtime event and underlying causes is available in https://nepton.fi/en/when-the-impossible-became-possible-and-what-happened-next article.
• Tue 25.10.2022 06:50 - All public services and features have returned to normal operation.
  • The backlog of offline records, integrations, and other automations, has been fully processed.
• Mon. 24.10.2022 14:12 - Service is available. We are still processing automated import and export integrations from backlog. You might be facing slowness in the service time to time.
  • Data has been restored from a backup made on Wednesday 19.10. at 9:15 p.m. Entries and changes made to the service after that time are not in the service, and therefore users must unfortunately be asked to make the entries again.
• Mon. 24.10.2022 08:46 - Service is online, but logins are temporarily disabled.
  The Nepton service is now online and we are processing the backlog of offline events, automations, and integrations has begun. Unfortunately, logins are disabled until overall load the platform has reduced.
  • We are truly sorry for the additional inconvenience caused. Our entire organization helps our customers and their main users in data processing and in all questions related to data processing. If we can help in any situation that occurred in the service, please contact us.
• Mon. 24.10.2022 06:40 - Nepton service will soon be operational

  • The problem situation started in connection with the normal maintenance of the data center storage solution and backup equipment, and concerned the equipment. It was not malware or a data breach, so the personal information and other critical data of customers and service users has not been threatened.

  • The storage solutions in our server center are organized by the world's leading provider of data center storage solutions. According to experts, similar completely unexpected failure situations are very exceptional. The disruption was massive, and the data stored in the storage solution was lost. International team of experts immediately started working to solve the problem.

  • The service is out of order until Monday morning. During the disruption, we have worked closely with storage provider's experts to restore data.

  • Due to the exceptional nature of the disturbance, we unfortunately cannot use the last-minute saved information of the disk system or its backup points, but the services are restored from a secondary backup solution.

  • The system data will be restored from the backup that was saved on Wednesday 19.10. at 9:15 p.m. Entries and changes made to the service after this time are not in the service, and therefore users must unfortunately be asked to make the entries again.

  • We are truly sorry for the additional inconvenience caused by a reason beyond our control. Our entire organization helps our customers and their main users in data processing and in all questions related to data processing. Service can soon be used normally

• Sun. 23.10.2022 18:00 - Bringing the services up begins

  • We are starting to bring services to production. We have been working with the provider of data storage provider. Because of the unusual incident we are unable to use the data stored on the storage or it's recovery points. We will return services from secondary backups. Services are brought back online during this evening and early hours of tomorrow. We will inform about this more in later updates. 

• Sat 22.10.2022 11:24 - Actions to solve the downtime
  • Nepton service is temporarily down. We are sorry for the inconvenience and are working with our data storage solution provider to get services up again.
  • On Thursday at 15:30 storage supplier performed normal maintenance for the solution. During this maintenance the fault-tolerant storage solution alongside with it's backup devices unexpectedly stopped working. The data saved for the solution were no longer available. According to the storage supplier, this is the first time they have seen this kind of totally unexpected situation. This is a major issue. International team of specialists started to work immediately to remedy the solution. They are working on the solution at this moment.
  • To recover the services the data integrity on the data storage solution must be checked. This integrity check was launched quickly on Thursday, right after the incident happened. It is difficult to estimate the duration of the inspection, because the tool that ensures the integrity of the data does not produce accurate estimates of the remaining duration of the inspection work. We have acted based on the instructions from storage supplier and are waiting for this data integrity check to complete.
  • Fri 21.10.2022 15:10 - Recovery from data center fault continues. Working closely with vendor crisis teams, faulty hardware has been identified and replaced. We are now validating and synchronizing all data to ensure consistency and integrity. This is a slow process and it is very difficult to give  completion time estimates, but it is proceeding smoothly. Unfortunately, it is unlikely that Nepton will be available again today, but we will work around the clock until it is, and will keep you informed.
• Fri 21.10.2022 11:10 -  Recovery continues, but it is now clear that Nepton will not be available until after noon today. We apologize for this exceptional disruption and will continue to update you on all developments here. A full statement will be issued later, but we wish to assure customers that this event has not been a risk to data security or privacy.
• Fri 21.10.2022 07:10 -  Bringing Nepton online again is taking longer than expected. We are working with our data center hardware providers to resume normal operations as soon as possible. The latest estimate is that services will become available by 12:00 today.
• Thu 20.10.2022 22:22 - The issue been resolved and services are being gradually brought back online. This will take some time, but we expect services to return to normal during the night.
• Thu 20.10.2022 15:26 - We are aware of a problem and have begun investigations.

Sun 16.10.2022

  Platform update

Date range Sun 16.10.2022 22:00 - 22:40
Affected services Core features, Platform interfaces (API), Single Sign On (SSO), Terminals

• Planned security update.

Tue 11.10.2022

  Currently our service is down. We are aware of the problem and have begun investigations.

Date range Tue 11.10.2022 13:25 - 15:38
Affected services Core features

• Tue 11.10.2022 15:12 - We are aware of a problem impacting many parts of the platform and have begun investigations.
• Tue 11.10.2022 15:38 - Problems in the service have been resolved and the service is working normally. We are investigating the cause of this problem in more detail.

Sun 18.09.2022

  Platform update

Date range Sun 18.09.2022 22:30 - 23:00
Affected services Core features, Platform interfaces (API), Single Sign On (SSO), Terminals

• Planned security upgrade.

Thu 25.08.2022

  Service degraded

Date range Thu 25.08.2022 13:15 - 18:10
Affected services Work shift planning

• Thu 25.8.2022 18:10 - Problem in service that in some situations affected logging in has been fixed. Also, the problem in service affects viewing and adding workshifts has been fixed.
• Thu 25.8.2022 15:51 - A problem in service also causes an error when logging into the service occasionally. This issue will be fixed as soon as possible with the fix for the previously mentioned work shift planning issue.
• Thu 25.8.2022 13:15 - A problem in service affects viewing and adding work shifts. We are working on to fix the issue as soon as possible.

Wed 24.08.2022

  Service degraded

Date range Wed 24.08.2022 15:38 to Wed 24.08.2022 20:09
Affected services Microsoft Login

• Wed 24.8.2022 20:09 - Issue was caused by a routing problem in core operator networks within Finland. Issue affected various Microsoft related services. Some customers were impacted during 14:57 - 17:27 time frame.
• Wed 24.8.2022 16:09 - Logging into Nepton service via Microsoft Login was disrupted. Microsoft Login is now back to normal. We are investigating the cause for the disruption.

Sun 21.08.2022

  Platform update

Date range Sun 21.08.2022 22:00 - 22:35
Affected services Core features, Platform interfaces (API), Single Sign On (SSO), Terminals

• Planned security upgrade.

Tue 02.08.2022

  Service degraded

Date range Tue 02.08.2022 13:00 to Wed 03.08.2022 22:00
Affected services Microsoft login

• Wed 3.8.2022 22:00 - We have confirmed from our logs that normal operation of the Microsoft login has resumed.
  • Although Nepton had complied with all recommended changes for the tightening of Microsoft Azure AD security posture during 2021, additional unexpected changes were made by Microsoft at the beginning of August 2022. We have now made the necessary adjustments and normal operation of the Microsoft login has resumed.
• Wed 3.8.2022 19:30 - The root cause has been identified and a fix will be applied today at approximately 22:00. The Service log has been improved to provide more details in the case of this kind of login problems.
• Wed 3.8.2022 13:00 - At this time most users are able to log in as usual again.
  • There are still occasional failures with Microsoft logins for some customers.
  • Users can continue to log in with Nepton username and password instead of using Microsoft login (Read more here)
  • After 18:00 today we will update the platform with extra logging that will help us isolate the problem and develop a solution.
  • We apologise for the continued disruption. We are doing our best to minimise impact to you.
• Wed 3.8.2022 09:45 - We are continuing our investigation into the cause of the problems and will make updates here as more information becomes available. At the time of writing, the issue only appears to affect a handful of customers. We apologise for the inconvenience and hope to have the situation resolved today.
• Tue 2.8.2022 13:00 - We have had reports of problems with Microsoft login and have begun investigating.

Sun 17.07.2022

  Platform update

Date range Sun 17.07.2022 22:20 - 22:50
Affected services Core features, Platform interfaces (API), Single Sign On (SSO), Terminals

• Planned security update.

Sun 19.06.2022

  Platform update

Date range Sun 19.06.2022 22:00 - 22:40
Affected services Core features, Platform interfaces (API), Single Sign On (SSO), Terminals

• Planned security update.