Hunting the black holes
The story started somewhere around 2003 when I was looking for a tutor of my master thesis. After some set of coincidences, I met prof. G.Wrochna and prof. L.Mankiewicz who led the PI of the Sky project. They had nearly no funding but heads full of great ideas. The idea was nearly crazy - observe all visible sky with a resolution of a few seconds in search of optical counterparts of the Gamma Ray Bursts (GRB). One of the mechanisms that cause such events is the formation of black holes. The events are extremely rare, just a few per day. And a lot of false alarms from planes, blinking satellites, meteorites, cosmic muons and detector noise. To detect such event a lot of data need to be processed every day, on-line. In remote observatories at the time, internet access was very slow, just a few kb/s. So the system had to be able to process GB of data itself in search of these rare events.
As a student, I didn't know that the requirements were tough. I had very little experience with professional electronics. I did some DIY work, was also repairing consumer electronics to have money for living during my studies.
We needed good detector sensitivity, high resolution - roughly 64Mega Pixels (remember what was the resolution of digital cameras 20 years ago?) The cameras available on the professional market were hell expensive . And such cameras had one essential drawback - the shutter. We needed to make a photo every second. Maybe every 5 seconds. This is a lot if one takes into account that the experiment lifetime needs to be a few years. Existing shutters were designed to perform 50k shots. In our experiments, they would break apart after a few days. We needed a lot more, something like 50 million cycles. No way to get such shutters from the market.
After a few weeks of work, I built the first prototype of the camera. It was based on 4Mpix CCD sensor from Fairchild, had USB2.0 interface (it was real speed daemon that times), Peltier cooling and awful aluminium enclosure that was carved by my brother at home with basic hand tools. But it was acquiring images. The shutter was my idea and was made of old hard drive head positioning linear actuator. It was behaving very well. In the picture below, the shutter resides under a thick aluminium wall that holds the lenses. The camera was built in my student hostel room, I remember the moment when I was spray-painting its components on the hostel corridor being poked by other students.
The sensor was installed on a dedicated board together with all electronics, CCD drivers, USB interface and FPGA. I used the very first Cyclone-series FPGA from Altera.
The camera had poor performance - high noise exceeding 50 electrons/pixel/second. We needed less than 15. It was caused mainly by the proximity of the digital electronics.
But this camera was working! It captured images. And the shutter was also working as expected.
My tutors from the University of Warsaw saw the potential of my work and organized a lab in the basement of the building on Hoża street in Warsaw. I got some basic lab equipment and a very good digital Tektronix scope which I used for many years.
My work was noticed by a professional team of scientists from the University of Warsaw. They designed new version of mechanics, cooling and better shutter
The shutter was made of two hard drive actuators to make the opening and closing process symmetrical and compensate for the "kick" that was present in the first prototype
The electronic circuit was split into two boards, but schematics got only cosmetic changes.
Then it started looking in a really professional way. Prof Wrochna installed the camera on his own roof to check it in action.
And got first sky images. It was remarkable success.
We quickly built second camera, guys from Warsaw University build dedicated robotic mount and we got such "beast"
During experiments we noticed that USB2.0 interface has several drawbacks - short distance and reliability problems. And DRAM chips I used were outdated. We decided to upgrade the camera to Gigabit Ethernet keeping the USB support. I spent whole summer 2005 in my lab located in my parents house designing and debugging new camera. I found some photos from that time...
It was not easy to get Gigabit Ethernet chip to work. I decided to reverse-engineer a PCI network card used in PCs. it was based on RTL8169 chip.
Here is the set-up I built. I designed simple carrier board with PCI slot and FPGA.
With the code of Linux driver, datasheet of similar chip I managed to make it running. The task was fairly complex - I had to build PCI master engine controlled by 8051 processor, PCI slave tied to the SDRAM controller which I also wrote. The Rx/Tx buffers were addressed by pointer areas (descriptors) so I had to add them as well in the FPGA SRAM.
The block schematic of the implementation is below.
Quite interesting solution was integration of the SDRAM controller with DMA and USB controller within single state machine. The circuit was able to continuously transfer the UDP frames with over 95MB/s speed. The 8051 needed only a few cycles to prepare next frame to transfer. All checksum offloading and UDP protocol handling was done in the FPGA. Only ICMP and ARP protocols were implemented in the 8051 firmware. I used the 8051 CPU embedded into USB2.0 controller. USB was also working as before. I utilised all 64kB of 8051 memory.
After I managed to make such set-up working, I designed the new PCB for camera.
In the meantime the project got some funding, I started PhD at CERN, the lab was moved to Świerk National Centre for Nuclear Research. The project was continued by my colleagues from the faculty. From time to time I visited the new lab to solve some issues...
In the meantime better robotic assembly was build.
We started using other CCD sensors from STA company which led to many issues because every sensor needed individual treatment.
Small series production of the cameras started.
In the meantime I finished the PhD contract at CERN and in 2008 started my own company, Creotech Ltd. The very first contract was production of electronics for new cameras as well as integration and tests.
Some of the cameras were installed in Chile in remotely operated observatory.
In September 2008 the Pi of the Sky system detected optical counterpart of GRB 080319B which was then described in Nature paper: https://www.nature.com/articles/nature07270
It was great success of the whole team.
Later on more cameras were installed in Spain.
The project website is here https://www.pi.fuw.edu.pl/
Some photos are here https://www.pi.fuw.edu.pl/project/final-system/piinta_site_jul2013/
In the meantime we developed (with my student Maciej Kwiatkowski) next version of the camera, with ARM processor and Linux, but I was not satisfied with its features. The network performance was very poor.
But it was a building block used to develop series of smart cameras with embedded video processing that were mass-produced and deployed in several shops to count the people.
The camera story continues, this is one of the recent Creotech Instruments SA achievements - the 64Mpix camera for Space Situation Awareness project.
The camera is based on ZynQ SoC, has vacuum CCD chamber, liquid cooling and utilises 100k$ CCD sensor. 16 such cameras are mounted to the telescope assembly. Very precise shutter was also built.