['AI', 'Engines', 'Supercomputers']
Blog Post #3 Understanding AI And Its Future
In Isaac Asimov's short story "Liar!", a robot is accidentally programmed to be telepathic. When it starts to realize the feelings of those around him, it begins to tell flattering lies. In the story, the robot is aware of it's programming error but doesn't let the researchers know to let them solve the problem themselves.
The story takes place in 2021, a short time from now, and already today we are seeing instances of ideological biases finding their way in AI. For example take BERT AI System, because of it scanning and learning from lots and lots of digitized information such as old books, maps, and records, centuries of bias are included in the result, such as it associating men more than women with the word "programmer". The data we give our programs is crucial and determines what valuations they make. In 2017, the reigning chess AI program was Stockfish, built with 100's of thousands of training examples. Google developed AlphaZero as competitor in chess engines to prove recent AI software improvements. AlphaZero taught itself chess, knowing only the rules, and by playing games against itself in random moves. It took 44 million games in nine hours for it to become the best AI chess program in the world. It then went on to beat Stockfish 28 times out of 100 and draw the remaining times. It examined a very low percent of positions-per-second, relying on teaching itself intuition instead using a tree-like structure to calculate variations, and an evaluation function to assign the position at the end of a variation a value. This is different from previous chess engines that relied on the minimax algorithm. For those not familiar with the term, this is essentially a kind of backtracking algorithm used in decision making, which finds the optimal move assuming the other player also play optimally (Tic-Tac-Toe, Backgammon, Mancala, Chess...). Two players are called maximizer and minimizer and one attempts to maximise the score for a given game while the other needs the score low.
This was back in 2017, and going by the fact that in 2019, computing speed doubled every 3.5 months, we could be processing 64-128 times faster today. The problem however is that we're running out of space. The architectural mode we use to build chips is down to nanometers and it is impractical to reduce them any further without reaching the atomic level. Quantum computing might bring about a new era, or we may see new materials used in the manufacturing process. 2D antimony is a recent innovation tested because silicon is no longer capable of doubling the packing of transistors on a chip. Antimony is a semi-metal with a charge mobility besting that of silicon and other semiconductors, making some hopeful.
The future it seems will also include Exascale computing, as three machines built by Cray are being readied for launch. These computers by definition are capable of at least one exaFLOPS, a quintillion, or 10^18 adds or multiplies per second. This is a thousand-fold increase over petascale supercomputers built in around 2008. 10/18/2019 suitably marked the day for the first annual exascale day, hoping to to acknowledge that "it really is the major driver for the future of our society and making the world a much better place as far as advancing science, building better products, improving health care for everyone... and also doing very unusual and fascinating thing like testing the impossible in the world" (Hyperion's Joseph).
Similarly, Intel just recently demoed a new GPU Architecture which will make its debut in the Aurora supercomputer at Argonne National Lab in 2021. The 'Ponte Vecchio Xe' data center graphics card is being built on the 7 nanometer scale, and enables logic-on-logic integration. PV will employ Foveros (die stacking), Embedded Multi-Die Interconnect Bridge (EMIB), high bandwith cache and memory. anandtech.com estimates it will have up to 2400 Aurora nodes. Aurora will top the scales with an estimated cost of $500 million, slightly higher than the current leader Summit at $200 million.
It looks like a bright future is ahead for Artificial Intelligence with ever more efficient hardware and software. Thanks for reading this blog! Check out my other blogs!