Commercial remote sensing has boosted a new revolution in traditional processing chain. During the development of OVS-1A and OVS-1B, we construct the main processing pipeline for ground and calibration system. Since these two satellites utilize colorful video imaging pattern, the underlying video stabilization and color adjustment is vital for end user. Besides that, a full explanation is given for researchers to shed light on how to promote the imagery quality from manufacturing satellite camera to generate video products. From processing system, the demo cases demonstrate its potential to satisfy end user. Our team also releases the possible improvement for video imaging satellite in the coming future.
The OVS-1A and OVS-1B video satellites (also known as ZHUHAI-1 01 and ZHUHAI-1 02 [
After one year running around, we feel honorable to reveal the main processing methods used, which may be different from general close range community, for end user.
In the past few years, mini satellite for commercial user is largely developed from overhead video imaging which can be recently dated back to typical SkySat [
Given a tight budget, Beijing Institute of Space Mechanics & Electricity (shown as BISME hereinafter) undertook the contract in 2016 from Zhuhai Orbita Aerospace Science & Technology Co, Ltd. (available at http://www.myorbita.net/), which must be finished in one year. Our engineers inherit some effective design from industrial applications and furthermore proposed some techniques to promote higher quality for standard product.
Name | Value |
---|---|
Area detector pixels | 4096 × 3072 |
GSD | OVS-1A: 1.98 m OVS-1B: 1.96 m |
Frame rate | 20 fps working at video mode; 1 fps working at footprint mode |
Quantilization Level | 8 bit working at video mode; 10 bit working at footprint mode |
Focal length | OVS-1A: 1536.2 mm OVS-1B: 1540.3 mm |
Spectral range | OVS-1A: 410 - 706 nm OVS-1B: 418 - 704 nm |
Weight | 12.93 kg |
Size | OVS-1A: 256.20 mm × 294.26 mm × 506.10 mm OVS-1B: 256.10 mm × 295.36 mm × 507.40 mm |
From the working mode in
To the radiometric calibration, a classical and uniform region is not practical any more [
A subset of real image covering Cape Town is given in
1) How to get high level radiometric calibration result for area device;
2) How to make change for Bayer formatted pixel to get an uniform image;
3) How to calculate the color transformation for true RGB;
4) How to implement video stabilization and approximately still video;
5) Geometrical processes for video sequences.
Before going to the main process, the linear model should be given for imaging pixel, and the reader can go to this section after knowing the challenge for future devices. The widely used is linear model [
Suppose the area scanner contains m × n pixels, 4096 × 3072 in this paper, digital number of any pixel in image B can be expressed as:
B i , j = K i , j ⋅ L e + C i , j , i = 1 , 2 , ⋯ , m , j = 1 , 2 , ⋯ , n (1)
L e = ∫ λ 1 λ 2 r ( λ ) R ( λ ) d λ ∫ λ 1 λ 2 R ( λ ) d λ (2)
where B ( i , j ) ―the DN value in image B;
L e ―equivalent radiance from calibration light;
K i , j ―gain for radiance;
C i , j ―offset for radiance;
R ( λ ) ―relative spectral response;
r ( λ ) ―transmittance of optical system;
λ 1 and λ 2 ―range of wave length.
In most cases, the non-uniform noise in image mainly comes from the gain and offset differences of every pixel.
From the calibration images, the normal no-uniformity coefficient can be calculated. Then stack the surface image at high and low digital value to generate two virtual images, a linear fitness of these two images modifies the initial NU coefficients. After that, we utilize the radiometric parameter to enhance image, generate L1 to L2 and video product.
Using the initial calibration data, the top of atmospheric no-uniformity coefficient for raw data can be drawn as:
D N L = K i , j ⋅ B L ( i , j ) ¯ + C i , j (3)
D N H = K i , j ⋅ B H ( i , j ) ¯ + C i , j (4)
B L ( i , j ) ¯ = 1 m ⋅ n ∑ i = 1 m ∑ j = 1 n B L ( i , j ) (5)
B H ( i , j ) ¯ = 1 m ⋅ n ∑ i = 1 m ∑ j = 1 n B H ( i , j ) (6)
Above processes fit response of every pixel to the statistical mean value, and generate the initial gain and offset coefficients K i , j and C i , j .
Specially, for Bayer image, there should be two normal approaches for RGB channels calibration separately:
1) interpolate Bayer to virtual RGB, calibrate RGB separately;
2) pick real RGB from Bayer, calibrate them and demosaic to get final result.
We found the latter one is a physical and strict data reconstruction method which generates better result. In one word, Bayer image cannot be interpolated to RGB before radiometric calibration.
After the launch, non-uniform noise still residues in real image, and it can be easily attributed to atmosphere or some other transfer jitter. Actually, with the help of our former work [
Excellent results can usually be achieved with the inexpensive, widely-available 24-patch X-Rite color checker. The CCM (color correction matrix) can be applied to images to achieve optimum color reproduction, which is defined as the minimum mean squared color error between a corrected test chart patches and the corresponding reference values, which can be the standard (published) values.
From the radiometric model, the R, G, and B values of each output (corrected) pixel is mathematically a linear combination of the three input color channels, Equation (7) shows the recommended 3 × 3 CCM A and its usage, where the P is processed pixel and O is the original color value:
[ P R 1 P G 1 P B 1 P R 2 P G 2 P B 2 ⋯ P R k P G k P B k ] = [ O R 1 O G 1 O B 1 O R 2 O G 2 O B 2 ⋯ O R k O G k O B k ] [ A 11 A 12 A 13 A 21 A 22 A 23 A 31 A 32 A 33 ] (7)
And also, 4 × 3 CCM usually works fine, please refer to [
Consider the launching time is near winter in China, the atmosphere is somewhat cloudy especially near the North China, we introduce the dark channel prior to remove the bad effect and maintain a comfortable visualization.
Due to the mechanical vibrations caused by the engine and the effect of turbulent
wind, the video is distorted and unstable. Therefore the video frames have to be stabilized for immediate object detection and real time surveillance. Video stabilization is a process to reduce the unwanted motion of the camera so as to remove the jitter and instability in the video frames [
Mechanical stabilization employs mechanical devices like gimbal to reduce the vibration of the structure holding the camera. However, this method is not able to reduce all the vibrations and is costly to implement. The Digital stabilization technique, however employs image processing based methods and can be used to remove the needless vibration that cannot be removed by Mechanical stabilization method. It costs less than the Mechanical technique as it does not involve complex hardware and requires only software to stabilize the video. Therefore digital stabilization is a better method of stabilization since it is inexpensive and easy to implement.
We noticed a parametric strategy in IPOL (www.ipol.im) with open source implementation [
For further approximately still video, we choose a registration method based on feature point to generate such product. SIFT [
Although the compression standard is common in close range community, we got response from end user the compression always damage the expert analysis for small object, especially the high profile H.264 standard (it will incur small blocks near the edge and is of various compression ratios).
Hence, we limited the compression ration no more than 10 for interested region (or quality test video) and no more than 30 in other regions.
Condition | SNR | |
---|---|---|
OVS-1A | OVS-1B | |
Solarzenith 70 degree and 0.7 reflective land | 39.925 db | 39.625 db |
Solarzenith 30 degree and 0.2 reflective land | 31.5 db | 31.575 db |
Solarzenith 20 degree and 0.05 reflective land | 26.275 db | 26.15 db |
After building RPC file using attitude and GPS data, we can construct the L2 product mapped into a cartographic projection and corrected from sensor and terrain distortions. The mean value of geometrical location error is 500m after system correction and calibration (small satellite without high level star sensor).
Mosaic products will be automatically processed in order to provide end user with an ortho-image of larger size, this is vital for multi footprint mode.
We do not pave the traditional knife edge pattern (high cost) but find some natural object such as road or building to test the MTF value, showing in
To quantify our video stabilization method, we carefully detect the matched feature points and calculate the residual error of every frame. In Figure12, it can be seen the average of error distance is reduced from largest 5 pixels to no more than 1 pixel in every frame after our processing, which prove its effectiveness (please go to
Method | Distance of matching points |
---|---|
SIFT | 2.65 pixels |
Parametric strategy | 1.93 pixels |
Our method | 0.86 pixels |
One thing should be kept in mind, this approach will not be used in video sequences with rare feature point, such as small ocean island, desert region.
OVS system is designed hoping to optimize the end user’s need. Its main image quality performances are ensured by appropriate processing line in end user’s production center (we worked on it for months).
Despite the success or progress, we noticed:
1) There are potential drawbacks to using color filter array (CFA) sensor such as Bayer filter, since CFA inherently lowers the resolution of the underlying staring sensor relative to a single spectrum image [
2) Video mode remote sensing lacks universal framework to guide how to construct the product, and end user usually cannot get the longitude or latitude from video product from any video player. We anticipate a full analysis to get an engineering product definition in the coming soon future.
After OVS-1A and OVS-1B, we successfully designed the TDI mode for area detector and extend the data picturing mode to traditional long strip.
Last but not least, Orbita has launched its newest video satellite with 0.9m resolution in 2018 July, and it can also be seen as another symbol for developing consumer remote sensing.
Thanks should be given to Senior Engineer Pan in Orbita for opening the data center to us.
This research has been partly supported by National Natural Science Foundation of China under Grant 61701023.
The authors declare no conflicts of interest regarding the publication of this paper.
Cao, S.X., Qi, W.W., Tan, W., Zhou, N. and Hu, Y.F. (2018) Main Processes for OVS-1A & OVS- 1B: From Manufacturer to User. Journal of Computer and Communications, 6, 126-137. https://doi.org/10.4236/jcc.2018.611012