2013: Cognitive Systems

Posted on by
Reply

Between IBM’s Cognitive Computing initiative, GE’s Industrial Internet, and Cisco’s Internet of Everything, one point is becoming perfectly clear: the push to digitally integrate every object in our modern existence is stronger than ever.

Problem: If it’s programmable, it’s hackable.

Corollary: If it’s connected, it’s pwnable.

No doubt these major companies have sophisticated schemes for keeping device security under control, but a paradigm shift may be in order.  IBM is nearing the mark on this: by drawing a direct analogy between digital systems and sensory systems, they’re unlocking a world of infinite potential.  It’s clear how this method applies to generating immediate commercial value, but how does it apply to security?

The security case is difficult.  Secure embedded Linux OS’s are abundant, yet how many of them employ a robust analogy?  How many of them can boast a sentient operating system?  None yet, but it will come around.

This year, and several years following, marks the rise of cognitive systems.  As an entrepreneur, I too will play my part.  A lot of folks think we live in the Information Age.  I’m saying the Information Age has not even yet begun.

Next up: Kybernetik und der Übermensch

An Account of Seemingly Disparate Endeavors

Posted on by
Reply

Some people know me as a software designer, others know me as a mathematician-in-training.  I’ve heard from some that I should stop wasting my time on one and just commit 100% to the other.  I claim that this would be a huge mistake, and here’s why.

In fact, these two things are more deeply entangled than many would think.  Here’s the crux of it: to write good programs and ultimately design good software, you have to have an intuition for and understanding of deep, complex structures and the relationships between them.  To do good mathematics, you have to have an intuition for and understanding of deep, complex structures and the relationships between them.  Skill in one area can easily overlap with skill in the other.

Based on this, designing software helps me understand deeper mathematical structures, and studying mathematics (especially matrix groups, complex analysis) helps me understand software designs.  Quitting one or the other would ruin this synergy I have going between them, and besides, at present I earn my bread through software anyway.

 

A slight misunderstanding

Posted on by
Reply

Often in discussions of artificial intelligence I see and hear the quote, “The brain does around X calculations per second.”  Usually this number is around 100 trillion.  Why?

This is presumed because the brain is said to have about 100 trillion synapses between all of its neurons.  By treating each synapse as a computational element capable of performing an action based on a stimulus, the brain is then modeled as “something doing 100 trillion calculations per second.”

There are several problems with this:

1. What is the nature of these “calculations” we’re talking about?  Is this simple addition, probability tables, or differential equations?

2. The language of “per second” could wrongly imply that the brain somehow runs on a constant master clock.

3. Are there additional layers of important information being exchanged beyond merely the synapses?

In more detail,

#1: Given the tendency to want to measure things in FLOPS (Floating Point Operations Per Second), I can see why this approach would be appealing.  It’s as simple as just counting how many “computational elements” are in the system, and then saying it can do that many calculations per second, right?  I’m led to think, well, no.  A FLOP is likely to be an extremely simple operation such as addition or multiplication.  Something more complex, such as linear algebra routines or probability functions, will require sophisticated code and hence numerous instructions/FLOPS to execute it.  The argument that “the brain does 100 trillion calculations per second and therefore we will have true AI when computers can do 100 trillion CPS” then is as useful as saying “a human is made of 150 lbs of matter so when you have 150 pounds of matter you’ll have a human”, or something equally ridiculous.  The number of calculations is not completely unimportant, but it is secondary to what kind of calculations are being done.  In the case of classical computers, as stated, complex instruction sets use up many operations to do their work, and so a raw measure of simple calculations isn’t very informative of the system’s overall capability.

#2: We’re used to think of calculations happening in a uniform, clock-like way because of the way our chips are designed.  The problem is, the brain, for as far as we can tell, processes everything asynchronously.  Each node is operating more or less independently of the others.  That’s not to say that classical computers won’t be useful in emulating brain-like mechanics, but modeling an asynchronous system with a highly synchronized one comes with complications that should not be ignored.

#3: This also ties in with #1.  Namely, due to the passing of information through electrochemical channels, there are additional layers of computation that, as of yet, have not been completely modeled nor understood.  The actual communication mechanisms of the brain could be simpler than we thought, or they could turn out to be vastly complex.  It’s anybody’s guess right now.  But in any guess, a direct conversion from calculations per second (as in the brain) to something like FLOPS (as in classical computers) is like saying “a machine that can add 10 numbers together in a second can also solve 10 high-order partial differential equations per second”.  With extremely clever software something like this may eventually be possible (perhaps a map from addition operators to matrix solvers or something like that), but for now this kind of crude conversion is wildly inaccurate.

I worry that a lot of people are buying this idea that once we get 100 TeraFLOPS machines we’ll somehow have an uber-AI.  Unless software comes a long ways, those who are counting on this idea may be very disappointed when this emergence doesn’t happen.

It is worth noting that a quantum computer used for AI would be a completely different picture – different from both classical computers and from the brain.  A behemoth of the quantum variety would be capable of things that neither an Intel i7 nor a human brain can do, but that is another discussion entirely.

Until next time, then.

The Best and Worst of Tech in 2012

Posted on by
Reply

The following is a list of what is, in my humble opinion, the best and worst technology and tech culture trends in 2012.  I’ll start with some of the best, go over the worst, and end with more of the best as well as some points for 2013.

THE BEST:

1. Aquaponics is taking leaps forward.  Vertical farming, hydroponics, and aquaponics are all picking up steam in the marketplace, especially as Kickstarter and IndieGoGo projects sprout almost daily.  I’ve long felt that we needed better ways to grow food, in urban areas or places with poor soil, and aquaponics has long been a viable solution.  It just hasn’t widely popularized until now.

2. Quantum physics often makes headlines.  I understand a lot of physicists would see this as a bad thing, given how badly the concepts are utterly botched, but I see the net effect as positive.  More people are curious about deep matters of physics now than ever before, as far as I can tell.  CERN has done a lot to drive this forward, but they are by no means the only player.

The more that quantum physics moves into early education, the better off the next generation will be.  So far, I’d say we’re off to a decent start.

3. Open, online education takes off.  Between Khan Academy, EdX, Coursera, and the countless other superb online education sites, anybody anywhere can now get a decent education despite all other factors in life.  This is especially true as it pertains to programming.  You can now learn nearly any programming language online, for free.  This has already changed education as we know it, and all signs point to its continuing to do so.

4. As already mentioned, crowdsourcing has changed a lot about how we think about projects, businesses, and community.  Especially awesome are the community spaces that are popping up, such as makerspaces, art co-ops, etc.  Despite a shaky economy and the worst national debt in our history, people are pressing forward and doing what they love.

The global scale of these projects is also impressive.  Now there are Kickstarters spanning multiple nations via their founders, demonstrating the platform’s ability to link people of common interest together.

5. Open source software is more prevalent than ever – and is now complemented by open source hardware.  From Hadoop to the Apache suites, Arduino to Raspberry Pi (whose open source-ness may be in question), CentOS to Fedora – open source is king.  This opens up numerous pathways for new development that can’t presently be imagined.

6. Cloud computing gets a foothold.  It’s taken a while, from my perspective, from when I first started hearing about the idea until it started to really take hold.  I think it’s safe to say that cloud computing has finally become a market standard.  Many would argue as to when that was exactly, but I’ll just leave it at “sometime in 2012″.

7. Chips are designed to consume less energy, and produce less heat.  This goes hand in hand with #6 above: people started to realize how much juice we were really taking up, and did something about it.  Not only is this good for businesses, but good for the planet too – in case you somehow forgot that pitch.

8. Parallel computing grows.  From Adapteva to Intel’s new 48-core designs, we’re finally starting to see significant advances in truly parallel processing.  GPUs have had their part to play in this as well, with suites like CUDA and OpenCL driving massive markets forward.

And now, some of THE WORST:

1. The patent wars continue.  There are no measures which can adequately describe how damaging this has been to business, innovation, and ultimately, progress.  Between Apple, Samsung, Google, Oracle, and Microsoft, it’s been fire and brimstone all year in the courthouses.  The only bright side is that patent attorneys are getting paid by the truckload.

One could write an entire thesis on this and barely scratch the surface.  For the purposes of this blog, suffice it to say that the effects of this onslaught will be felt for a long, long time (due to precedents set, laws made, etc.).

2. This follows naturally from #1: the US patent system still does not have comprehensive support for software or algorithms.  Patenting anything in this vein reduces to invoking the kind of convoluted sophistries that only those fluent in legalese could produce.  Acknowledging that software/algorithm patents are a touchy issue, the least we could do is disambiguate what can and cannot be patented.  Without clarity, good business cannot be done.

3. Everything starting with “i”.  iCar.  iWork.  iBot.  iGarage.  It’s gotten both ridiculous and annoying.  In the first place, mimicking Apple is not always the wisest thing to do.  And secondly, it really couldn’t be that hard to be more creative with your names, could it?

4. Everything is “smart”.  SmartCard.  SmartGlass.  SmartTires.  SmartSmart.  SmartSmartSmart.  This too has gotten utterly ridiculous.  Putting “smart” in front of a generic product name does not say anything about the product or the company.  I could make a “SmartStick”, and that wouldn’t change what it most certainly is: a stick.  Likewise, a ‘SmartCard’ is still just a card.  Better to make some indication of what the thing actually does or is useful for.  Even “MetroCard” is fine: it’s a card that you use for the metro.

5. Apps.  I don’t have any problem with apps themselves, but the feverish craze that has everybody chattering like caffeine-injected chipmunks about the latest apps drive me crazy.

Of course mobile technologies have added tremendous value to many aspects of our lives.  I get that.  And of course it’s exciting – they’re nifty gadgets with lots of even niftier sub-gadgets (apps).  But the marketing craze and the lingo surrounding the whole thing has gone far beyond acceptable.  (Too many times when I tell people that I develop software do they just default and say, “Oh, so you make apps.”  No.  I do not make apps.  Software is infinitely more than apps.)

To celebrate the insanity, I came up with some great product names this year: iSmartApp.  iAppSmartSmartApp.  iLookSmart.  My personal favorite, a tech company: iSmartSolutions.

Buzzwords are always annoying, but I promise I won’t complain about any more of them in this posting.

6. Still, nobody seems to have a clue as to what to do about the cybersecurity problem.  There’s a lot of panic presented by the media – no surprise there – but there is also reason to be concerned.  As of yet, nobody in particular has stepped up to the plate to offer a feasible fix, however temporary.  As devices grow closer and closer to users (and eventually implanted in the users), this is hugely important.  Quantum-encrypted interconnects would be a nice start, and fortunately there is lots of active research toward that, but it may be taking too long.

More of THE BEST:

9. Space technology is beginning to bloom in the private market.  By now, almost every tech person knows this, but it’s still good to realize how important this is for the human race.  Getting out into the stars has been the ultimate dream, and it is soon to be a reality.

10. 3D printing has also seen huge improvements.  As the precision and the number of materials one can work with continues to grow, this family of techniques is sure to transform manufacturing forever.

11. The world didn’t end after all.  We’ll all have 2013 to develop more amazing technologies and change the world.

2013 promises to be an industrious year.  The economy is going to come back stronger than ever, new tech is right on the horizon,  education is being transformed worldwide, and we’re starting to learn from our mistakes.  We have everything to hope for.

“How to Create a Mind” Review

Posted on by
Reply

I’ve just finished reading Ray Kurzweil’s new book, “How to Create a Mind”.  In it I found a wealth of good information, especially in the form of thought experiments.

Kurzweil’s latest work ties in a mass of data about the brain, pattern recognition, and his own experiences, creating a sort of roadmap to creating strong AI.  The account is clearly written and concepts are well explained.  He includes some interesting research, perhaps most intriguing are the experiments with split-brain patients, illuminating more subtle aspects of consciousness.

The grand theory presented in the book, the Pattern Recognition Theory of Mind, has some nice features.  It promises completely asynchronous processing to emulate the brain’s same ability, as well as uniformity of elements.  This uniformity is part of what enables arbitrary regions to be configured to process different types of information, given sufficient exposure to their respective type of data.

Kurzweil’s formulation of hierarchical pattern recognition seems to stem almost exclusively from Hidden Markov Models, specifically of the hierarchical variety.  While these models are indeed useful for many applications, missing throughout the book is an explanation of any explicit role of time.  In Jeff Hawkins’ “On Intelligence”, temporal patterns take a central role in the theories presented, distinguishing it from most traditional machine learning designs.  By contrast, Kurzweil’s PRTM (Pattern Recognition Theory of Mind) does not take time directly into account.  We’re left to assume that temporal patterns are implicit in the changing of spatial patterns, though some definite remark on that would have been helpful.

Most of the book’s real value does not come from detailed algorithmics or mathematical ingenuity, but again from the deep and illuminating thought experiments presented.  Kurzweil has a way of exposing subtle relationships in concepts that no other author can, save Marvin Minsky (another personal favorite, who was a mentor of Kurzweil’s).  The book delivers a powerfully enlightening look into the intricate world of pattern recognition, and presents fascinating a viable avenues of exploration for making intelligent machines.  Anyone who is interested in the brain, AI, robotics, or just technology in general should definitely give this a read.

Netbeans for C, C++

Posted on by
Reply

This is just a quick note:

In my limited experience, I’ve nonetheless tried the full gamut of IDEs.  One that I keep returning to is NetBeans.  When I was working with it with Java it was a pain, but I think that was all Maven’s fault.

Let it be said that the Netbeans C and C++ version is superb.  Usually you lose so much time fiddling around with IDEs and getting the errors fixed.  This distribution is simple and clean, and keeps the option of all of Netbeans’ features and ‘candy’.  My prototyping process using this has increased 5x, which is exactly what an IDE should do for you.

I’m still using IntelliJ IDEA for Scala purposes, but for C and C++, Netbeans takes the cake.

BNS Project

Posted on by
Reply

I’ve begun the Binary Neural Sequencer project on Assembla, though it will soon get its own website.  This is an open source AI framework being written in: Scala, C, C++, and eventually some others as well.  Some Clojure may assist with some functions with the Scala version.

The C version will be focused on robotics applications, so it will be geared toward embedded applications.  The goal is to make it compatible with Arduino and other microcontrollers, as well as the usual family of Intel and AMD chipsets.  A version for mobile chipsets may appear in the future, depending on who joins on the project.

The Scala version will be focused on Web applications, particularly media-rich content.

The C++ version will have a unique focus for CAD software, esp. as it pertains to space engineering – more on that some other time.

Other eventual versions may be developed for application-specific needs.

Our work will be done on Assembla, and on stable releases we’ll send the work to a public server for download.  All the code will be on the Common Public License.

I figured this distribution effort would have more of an effect than just putting it on this website.  Also, I’m not the only developer on the project anymore, and so the project is even more public now.

As we progress, I’ll continue to use the various versions in other projects of my own, but the source code will be available for anyone who wants to try it.  Deep Learning has set a good example of how open source AI software can change business.  The Binary Neural Sequencer project is aimed to add even more options to powerful open source AI software, for use in robotics, business, astronautics, bioinformatics, engineering, and many other industries.

For now, the project is in development.  I’ll post links to the downloads when we have our first release.  I’ll look forward to receiving feedback on the first release.