This is not a Drill! Responding to a Cyber Attack 

Briefly describe the initiative/ project/service; please include your aims and objectives

This nomination recognises the impact of the Council’s ICT and Digital Strategy in positioning the Council to successfully respond to a recent cyber attack.

In 2020 the Council committed to deliver a new ICT Strategy and Technology Roadmap with cloud technology at the centre of achieving a flexible but highly secure environment for the storage of data and digital property. The adoption of cloud, and the ability to provide secure authentication for those using it enabled new opportunities to join up, collaborate and share data safely. Members of the ICT team were part of this this strategic journey, leading the projects that made our cloud adoption rapid and successful. It was because of this and their knowledge of this strong strategic foundation, that when the first signs of a potential cyber-attack were identified, the council was well positioned to respond.

In the early hours of the 17 August, an unidentified ‘threat actor’ used Malware as a Service (Maas) to coordinate an attack on the council’s systems. This service can be purchased via the dark web and provides a set of scripted instructions and the tooling required to coordinate attacks such as this. Post incident investigation has determined that between 3am – 6am on 21 August, 29.28GB of Council Data (117k files) was exfiltrated by the attacker to a cloud hosting storage provider based in New Zealand.

As the council activated its emergency planning processes and implemented business continuity arrangements, which included the immediate invocation of its Cyber Security Support Provider – Communicate PLC, the ICSIRT and Communicate were quickly able to identify the threat and act upon the source of the attack. During the initial response period, the ICT team operated continuously, including during the night to monitor activity, put in place workarounds and re-build critical systems, despite the intensity of this work and the strain it placed on them, they all maintained a sense of composure and calm which allowed their efforts to become quickly effective.

After further forensic analysis completed, it was found that the incident had been successfully contained by 21 August, with a recovery plan implemented. Whilst the attack resulted in significant internal organisational disruption with many staff and teams experiencing limited access to key lines of business applications, the team quickly restored these systems allowing business as usual to continue with minimal impact for our customers.

What are the key achievements?

In a cyber incident, effective communication is vital throughout, to coordinate action and ensure quick decision making based on credible intelligence. The team consulted with multi agency stakeholders including the Department for Levelling Up, Housing and Communities, the Information Commissioner’s Office (ICO) and the Department for Work and Pensions (DWP), as well as other Local Authorities, the Police and NCSC. The effectiveness of the team was further demonstrated as we were able to report the data loss to the ICO in line with expected timescales and, further to extensive internal investigation, the ICO have confirmed that they are satisfied with the outcome and will not be pursuing enforcement.

Internally, the team reported to the senior leadership Strategic Command Group (SCG), and Incident Response Team (IRT) to ensure timely and relevant updates to staff, stakeholders and the wider public.

During the first week of the incident, the ICSIRT enabled the following key actions:

• Deployed enhanced protection tools and 24×7 monitoring services across servers and devices (Surface Pro, laptops etc).
• Technically and physically coordinated a full in-person password reset exercise across the council workforce and elected members.
• Identified servers, end user devices (such as laptops) and network account assets (like Citrix), as being linked to the incident, and removed them from the network for examination, rebuild or destruction.
• Took precautionary steps to protect council systems by immediately disconnecting other servers from the network, whilst containment and eradication was enacted.

Post incident investigation suggested that the attack may have been a precursor to a more substantial cyber event with potentially greater implications in terms of data loss and financial demands. The team’s response and recovery procedures limited this for both the organisation and service users, preventing further escalation.

External partners and crime agencies praised the work of the ICSIRT highlighting their dedication, professionalism, knowledge and leadership as key factors in the successful response and recovery to the incident. Most organisations take many months, some years, to recover from a Cyber-attack, the council recovered fully in ten weeks, with the substantive recovery within eight weeks.

The ICT team demonstrated the highest professional and technical standards in understanding the ICT strategy to position the council at the leading edge of digital development, and enabled the associated continuity plans to be effective and credible, which when tested in an organisational emergency, and supported by this exceptional team showed exemplary commitment and dedication to protecting the council, its staff and service users.

How Innovative is your initiative?

Firstly the strategy set out a highly advanced approach to Cloud technology, not seen in many other local authorities, which positioned the organisation well to respond to the cyber incident.

Over the past four years all new Council services have been implemented on or migrated to the cloud (or hosted platforms) rather than the traditional on-premise deployments in our own data centres. Instead of ICT having to drive this move alone, it is being actively driven by our system owners and their associated suppliers who recognise the benefits that cloud computing can bring. We operate systems across three key resilient UK based cloud platforms including Microsoft Azure, Oracle Cloud and IBM Cloud, our key area of specialism is and will continue to be Microsoft Azure as this is the most dominant of the cloud platforms we use and contains the bulk of the cloud services we deliver, for example Microsoft Office as part of the M365 suite, Microsoft SQL as the Councils principle database technology and applications systems based on Microsoft Windows. This has and continues to be hugely advantageous as we are able to quickly deploy systems and manage services ourselves, this enables us to make significant savings over the use of third party vendors who can provide similar services at sometimes increased cost, but more than this it allows our system owners to have options when considering the way their applications will be delivered as they move through their natural life and upgrade cycles.

In terms of information security, the strategy applied a balanced approach to risk management when adopting new technical architectures, services, platforms, and methods of working. Innovation must be balanced against the risk of exposing the Council to significant risk in terms of reputation, public trust, and compliance but at the same time recognise that we need to trust our employees when using these services. The Council aligns its policies and procedures to the guidance issued from the National Cyber Security Centre (NCSC), is an active member of the North West Warning, Advice and Reporting Point (NWWARP), has its own IT Policy and Regulation Group (ITPARG) and is accredited under the International Standard for Information Security (ISO 27001) and Data Security Protection Toolkit (DSPT).

The IT provision includes a range of protections including anti-virus, malware, anti-spam, web filtering and mobile device management (MDM) technologies to keep those using Council systems safe.

What are the key learning points?

What worked well?

• ICT & Digitals continuity plans, which are reviewed under our ISO standards by external auditor, and which we closely followed. These are ISO/IEC 20000 and ISO 27001 audited by Lloyds Register Quality Assurance (LRQA).
• Adoption and Migration to cloud of around 60% of our systems, which simply were disconnected but otherwise unaffected.
• ICT & Digitals deep technical knowledge to be able to work with the forensic partner we engaged.
• ICT & Digitals knowledge of our systems (many written by us), to work with service leads to implement their own business continuity plans, put in place workarounds, or recover them ourselves.
• Our tight knit ICT community in the LCR and beyond, who offered to support us, and brokered messages to those who had disconnected us, this was invaluable.
• High level support and coordination from both the Strategic Command Group and Incident Response Team.
• Clear Communications to our partners and agencies during the incident.

What would you do differently?

• With the exception of now having CrowdStrike, SIEM and enhanced monitoring in place, not much as we recovered well as evidenced by the short recovery period of all of our services, many of which came back within a few weeks, this showed our plans worked, that said the following things are being considered.
• We need to review service continuity plans as many of these are not good enough for when an incident occurs.
• Ensure that regular testing of service continuity plans is undertaken.
• Complete our adoption of cloud which would have seen an even quicker recovery to BAU.
• Collate government contacts as they are all different, all have different procedures, and some don’t consider the situation (DWP in particular), when asking for evidence of eradication, some settled for a statement of containment, whilst other appear to have an industry behind them waiting for an attack to happen – DWP have said they will review their processes following this and we have suggested they look at consolidating ALL contact into one (a bit like Tell Us Once for Cyber).
• Have in place a dedicated Cyber and ICT Security team.

What are your key learning points?

• Keep calm and don’t panic
• Our defences are only as good as what we have in place, ours like many surrounding and further afield LAs were just not good enough.
• To the previous point, ensure as we have now (and were procuring before the incident), a proper external Cyber, Forensic partner to call on and support during and beyond the incident.

Would you recommend others do?

• Review your Cyber Security posture.
• Engage an external partner to advise you.
• Implement a SOC and SIEM if you don’t already have one in place.
• Act on the advice you are given or if you don’t then mitigate it in another way if possible.
• Ensure your ICT plans and especially service continuity plans are in place and are credible, and TEST THEM REGULARLY
• As above ensure you have enough Cyber security expertise in your ICT teams and keep them trained and up to speed.
• Attend one of the sessions we will be running in the New Year through various forums to gain a deeper understanding of what happened to us and how we dealt with it.