Anatoly Solovyev, RAS Academician, Deputy Director, Geophysics Center of the Russian Academy of Sciences
The development of terrestrial and space systems of global monitoring and the introduction of modern equipment that provides high frequency registration of geophysical parameters have led to unprecedented growth of the volume of registered data in the Earth sciences. Effective transfer, storage, processing and transmission of complex sets of geophysical data require adequate methods and mathematical algorithms. This is particularly important in the study of the interaction between the Earth and the Sun since the effects that appear in the Earth’s magnetosphere under the solar influence can have a significant negative effect on the modern technology infrastructure.
The latest findings in the area of system analysis and the work associated with the so-called big data provide us with an tool to solve the problem of processing of great amounts of geophysical measurements. Methods based on such approach are universal in nature and can be used on various types of data. In the area of geomagnetic research modern methods of system analysis and artificial intellect provide tools for automated multi-criteria detection of extreme events of various nature. Complex analysis of ground and satellite data gives us an opportunity to build a highly precise model of the Earth’s magnetic field elements, which is very important for solving a number of scientific and practical tasks.
Geomagnetic field registered on the Earth’s surface as well as in the near-Earth space can be divided into internal and external field. Processes that are taking place in deep inside the planet serve as the sources of the internal magnetic field. Changes in the internal magnetic field take place over long periods of time – dozens and hundreds of years (century variations). The external magnetic field is formed by the complex and highly variable spatial structure of the electric currents in the magnetosphere and ionosphere of the planet that are driven by the solar influence.
Geomagnetic activity is driven by relatively short-term variations of the external magnetic field caused by the solar activity. The effect produced by magnetosphere currents on Earth is observed as the magnetic field value deviations from the neutral level in time scale from seconds to dozens of hours. Increased level of geomagnetic activity and geomagnetic variations of extreme amplitude can pose danger for various technology systems (electric grid, pipelines, satellites, etc.). Thus, geomagnetic monitoring and the detection of specific geomagnetic effects in real time are an important task in the cause of providing technological safety. Extended observations of the changes in the internal magnetic field are also important for the understanding the causes of its evolution.
Continuous measurement of the geomagnetic field parameters is carried out at special observatories located around the world. Modern magnetic observatories are high technology facilities that have been in operation for a long period of time that provide highly precise ongoing registration of the magnetic field, which register both the century-long as well as short-term variations. The international network INTERMAGNET — International Real-Time Magnetic Observatory Network is the most developed coordinated network of magnetic observations that provides top quality data. At present time the network includes about 140 observatories all over the world.
In the last few years Russia has also achieved significant results in developing its ground magnetic observations. With the support and supervision of the Geophysics Center of the Russian Academy of Sciences – one of the leading research organizations in this area – existing observatories have been upgraded to meet the international standards. As a result, in June 2016 the St. Petersburg observatory was officially included in the INTERMAGNET network. The Geophysics Center also took part in the development of the new magnetic observatory Klimovskaya in the Archangelsk Region. The Geomagnetic data center of the Geophysics Center receives data from 13 Russian observatories, of which 9 are a part of the INTERMAGNET network.
Present time magnetograms that the INTERMAGNET network observatories transfer have the status of preliminary data. They can contain man-made noise and omissions but provide data with minimal time delay. Magnetograms that have undergone the complex and labor intensive correction procedure to clear all the artificial noise receive the final data status. Preparation of the final data for a specific observatory for one year is usually performed manually and can take up to two years. In order to speed up the preparation of cleaned up data a new type of magnetogram was introduced a few years ago – it was called quasi-final data. In their characteristics such magnetograms are similar to the final data, but require a lot less time for preparation. Quasi-final data are formed on-site at the magnetic observatories. Highly skilled specialists perform the preparation manually.
The hardware and software complex developed at the Geophysics Center automates and accelerates the procedure of the ongoing collection of magnetograms from the Russian observatories and the preparation of quasi-final data and final data. Such results are obtained by using modern algorithms that include elements of artificial intellect. The majority of operations are performed in quasi-real time, which provides the opportunity for a real-time evaluation of the magnetic activity necessary for developing precise forecasts. The newly developed hardware and software complex is the first system to prepare quasi-final magnetograms. It also serves for detection and multi-criteria classification of extreme geomagnetic phenomena in automatic mode. Introduction of such smart systems sets apart the quality of the Russian observatory network as compared to the global level. It is noteworthy that many of the INTERMAGNET observatories still analyze magnetograms manually, which causes significant (up to two years) delays in preparing the final data.
Another important advantage of the newly developed hardware and software complex is the ability to combine in one system geomagnetic data obtained from various sources.
Along with the ground observatories low orbit satellites provide homogeneous global coverage of magnetic measurements. The current satellite group Swarm engaged in the monitoring of the Earth’s magnetic field was launched in November 2013 from the Russian space launch station Plesetsk. The satellite mission of Swarm consists of three identical pieces developed by the European Space Agency (ESA). The main goals of the mission include measuring of the characteristics of the magnetic field for the study of the processes taking place in the Earth’s core, mantle, lithosphere, oceans, ionosphere and magnetosphere.
Due to the fact that Swarm data is processed by the hardware-software complex makes it a leading edge instrument for coordinated processing and joint analysis of ground and satellite data, which expands the number of areas this data can be used for.
The hardware-software complex is the core of the analytical center for geomagnetic data of the Russian segment of the INTERMAGNET network. The complex is based on the latest developments in the area of geophysical processes monitoring and intellectual data analysis. The complex transmits, stores and performs multi-criteria processing of the data coming from the ground observatories and satellites of the Swarm mission. The complex is built according to the modular principle and is a flexible system with a great potential for expansion of its functionality. The technological approach used to develop the complex offer an opportunity for easy replication, thus turning it into a standard solution.
The main functions of the complex include the following: automatic downloading of initial ground and satellite magnetic measurements; automatic filtration of the data coming from the observatories to remove artificial noise and data verification; recognition, classification and encoding of data related to extreme geomagnetic phenomena; model calculation in online mode.
The original and processed observatory magnetograms, satellite data, as well as the analysis results and model calculations are stored in a single relational database. Thus, users of the complex have greater flexibility in formulating their inquiries and have user-friendly flexible interactive access to the whole body of data stored in the database. Such an approach has been used for the first time and has no analogs in geomagnetic data analytical centers abroad.
The system offers a wealth of opportunities for the visualization of geomagnetic data, including the use of modern projection equipment with spherical screen.
The concept that serves as the basis for the system developed matches the modern paradigm of the information technology development in the area of big data processing. The complex speeds up the process of obtaining reliable data about the Earth’s magnetic field. The combination of data obtained from various sources – from the ground and the satellites – provides a diversity of collected data and increases the volume of our knowledge about the processes taking place on our planet. The complexes functionality makes it an extremely popular tool for experts and government representatives interested in evaluating and reducing the risks associated with extreme geomagnetic phenomena.
The hardware-software complex was developed in 2014-2016 within the framework of the project called The Development of innovative technology and creation of experimental sample of a hardware-software complex for monitoring extreme geomagnetic phenomena using ground and satellite data”.
