Upon the ‘Growth of Knowledge’, and ‘of Ignorance’

(28-12-2005)

The bottom line is: only the Growth of Ignorance can match the Growth of Knowledge. It does not matter how the problem is approached; by sheer numbers of books, articles or web pages. The problem of knowledge is not the production of it but its consumption. The number of books, articles and web pages in science, arts and humanities are rocketing. But who can catch up with it? Nobody can inevitably, so other options are required. (Very soon!)

The ‘Growth of Knowledge’ usually refers to the grand epistemological debates on paradigms (Kuhn) and research programmes (Popper). Let us leave that for what it is and look at that growth from a down to earth view.

Since some decades the growth of knowledge is also referred to as exponential. Alvin Toffler (‘The Future Shock’, 1970) pointed out that even specialists already admitted that they lacked the time to keep up. That was a time without the internet as we now know it. It was a time in which article writing was a yet spreading common practice and a time in which researchers and scientist distinguished themselves by writing books. However, the number of books, articles and web pages have rocketed at different velocities since. The number of internet pages seems to double every six months. Was that number end 2005 some 600 billion pages (yes, billion), imagine the number in 2010 !

Though recent figures are missing, the year in which 1 million articles in sciences, arts and humanities are produced is near; at the moment (end 2005) the year’s production is probably something above 860.000. What will be the number in 2010? As for books? Perhaps the multiplier is no longer 25 years but 10, meaning that from 2000 to 2010 as many as books may be published as in the preceding 2000 years.

Who can catch up with these numbers? The point is that everybody is ‘sending’ information but there is serious doubt if the receiving parts and numbers can match this ‘growth of knowledge’. Who can ‘receive’, “consume” and integrate such piles of new information with the existing bodies of knowledge in whatever specialised topic? Year after year? Inevitably, nobody can. If it does not match, than the ‘Growth of Ignorance’ matches at least in degree, the ‘Growth of Knowledge’. Furthermore, it is at least plausible that the ‘in degree’ is also growing too, in time, to exponential levels.

No, this is not about the fashionable ‘innovation’ debate that wanders through the ‘Abendland’. This runs way deeper, to the core of knowledge; ‘Morgenland’ faces the same problem. Yes, there must be more ‘readers’ who indeed devote time to reading and to understanding. But that is not enough. Other options must be developed too, to cope with the collective inevitable too limited scholarly abilities in ‘Abendland, Morgenland und Mittagsland’. Can expert systems help? Really? Who knows, time is running short. And the longer it takes, the more information will have to be fed into such a system; outdating it before ever becoming operational.

Lets see some details; first level. Regarding web pages, there were 600 billion in November 2005. As it takes only six months to double that number the estimated growth per year is some 2400 billion web pages; scientific ones and non scientific ones in sum. And mind you; a web PDF of a 100 pages counts as one web page.

The yearly growth of scientific articles is some 1.5 million each year (so far). That approximates the yearly total number of new book titles, only of which a proportion is of scientific nature. Early 2001, the total amount of book titles doubled every 14 years and evidently, that reproduction rate is speeding. All in all, it seems to boil down to the statement of dr. Bontis with his expectation that by 2010 the total amount of accumulated codified databases of the world, which includes all books and all electronic files will double twice a day. These figures and others are listed in the table below.

Though not all data was retrievable or available so there are some blanc spaces remaining. Nor was it possible always to distinguish scientific production within the total amount of information. The picture however is clear and so is the challenge.

How does mankind cope with that amount of information? Expert systems that codify all knowledge into weighted relationships between dependent and independent variables? And adapting the weights in time perpetually? Perhaps.

Disregarding methodological points like comparing scientific and non scientific material or only counting Anglo-Saxon production at a global stage and disregarding China, Korea, Taiwan, Russia and the Spanish speaking production centres of knowledge, it is clear that some intelligence will be needed to sort things out; science from nonsense to start with. And next; fabricating bodies of validated knowledge. Preferably dynamic, adapting to and absorbing all new relevant data into its knowledge base. Knowledge not know is as valuable as prehistoric ignorance. Perhaps Twenty-first Century Mankind will be looked at –in hindsight- as ill knowledgeable cave dwellers. So, lets do some reading.

Domain	number	Source	Yearly Growthi	Source	Time to Double	Source
Web pages	600 bln	1a	In 2006; ± 2400 bln	1b	* 6 months	1c
Articles		2a	1.5 mln	2b		2c
Books(titles)		3a	1.6–2 mln	3b	* 25 years (1950-75) * 33 years (2000) * 14 years (15-5-2001)	3c
All codified databases, all books & digital files		4a		4b	* 7 years in the seventies twice a day in 2010	4c

*)

- 1a-Web pages, total amount (nov.2005): http://businessnetwork.theage.com.au/articles/2005/11/18/3491.html.

- 1b-Web pages, yearly growth Conservatively estimated from 1a: 2.400 billion

- 1c- Web pages, time to double www.cs.indiana.edu/classes/b669-rawl/node4.html)

- 2a- Articles, total amount: pm

- 2b- Articles yearly growth: (juni 2005): see pg 33, next to the photography http://www.eahil.net/newsletter/journal_2005_vol1_n4.pdf;

- 2c- Articles, time to double: pm

- 3a- Books, total amount : pm

- 3b- Boks, yearly growth:

(juni 2004: http://www.acls.org/cyberinfrastructure/cyber_meeting_notes_june.htm)

- 3c- Books time to double: Brian Quinn; The Economist Nov.11, 1995:

* 1950-75: After book print was invented (ca 1440), it took some 300 years before the amount of information (in the Western modern world, I assume) was doubled. Between 1950 and 1975 as many books were published as in the 500 years after inventing the book print principles. (Source: Brian Quinn; The Economist Nov.11, 1995 in Managing Knowledge: Building Blocks for Success, 1998, G. Probst, S. Raub, K. Rombart; in the Dutch edition of 2000, pg 14.)

* 2000: pg 72 of http://aepp.net/Proceedings2003.pdf (9-11 Oct 2003) “Today, the number of books published doubles every 33 years (Hanka & Fuka 2000). Equally we know that more information was published in the last thirty years than in the previous five thousand years (White & Dorman 2000).

* 15-8-2001 http://piano.dsi.uminho.pt/grupok3/bibdig/WersinDaniela-2001.pdf; … the number of books in libraries is doubling every 14 years the amount of knowledge which is available World Wide is doubling every. five years

-4a- Databases, total amount: pm

-4b- Databases, yearly growth,: pm

-4c- Databases time to double: at pg 72 of http://aepp.net/Proceedings2003.pdf (9-11 Oct 2003) Dr. Bontis of the Institute of Intellectual Capital Research suggests the total accumulated codified databases of the world, which includes all books and all electronic files, doubled every seven years in the 1970s. Dr. Bontis theorizes this database will double twice a day by 2010 (Bontis 2000).

==

Lets see some more details; second level.

In 2005 the number of web pages exceeded 600 billion; only some 40 per cent is commercial. Passion and duty spin the rest. Kevin Kelly, Friday, 18 November 2005 (Source: http://businessnetwork.theage.com.au/articles/2005/11/18/3491.html.

In January 2000, researchers at the NEC Research Institute and Inktomi estimated that the Web had more than 1 billion unique pages (Source: http://www.thememoryhole.org/crs/RL31270.pdf).

The number of webpages is doubling even faster. If the web keeps doubling every six months or so, then by January 1st, 2000, there could be 6.4 billion (Source: www.cs.indiana.edu/classes/b669-rawl/node4.html)

There is more:

* number of websites: http://news.netcraft.com/archives/web_server_survey.html

* languages at the internet: http://global-reach.biz/globstats/index.php3

* usage of internet: UNESCO PRESS No. 2005-133/2; Paris Nov. 3th, 2005 "Only 11 percent of the worlds population has access to the internet and 90 percent of those connected live in industrialized countries." www.nieuwsbank.nl/en/2005/11/03/f055.htm

* users by country March 2004: http://netforbeginners.about.com/cs/technoglossary/f/FAQ3.htm

* a photography of the internet: http://noorderlicht.vpro.nl/artikelen/15127569/

* up to 99% of the internet is considered ‘invisible’ http://www.planet.nl/planet/show/id=112030/contentid=510676/sc=26efb7

As for articles in science, arts and humanities? In 1980, a number of 448.000 were added to loads of them in the libraries. That yearly number increased too, to some 600.000 only 15 years later (1995) to 742.000 in 1999. (Source: RASCI-data; numbers for 1980, basic data for 1th & 2th European Science & Technology Indicators report (page S-53); numbers for 1995 and 1999, see CWTS Third European Science & Techology Indicators report (page 285,fig.5.2.8)

Books: After book print was invented (ca 1440), it took some 300 years before the amount of information (in the Western modern world, I assume) was doubled. Between 1950 and 1975 as many books were published as in the 500 years after inventing the book print principles. (Source: Brian Quinn; The Economist Nov.11, 1995 in Managing Knowledge: Building Blocks for Success, 1998, G. Probst, S. Raub, K. Rombart; in the Dutch edition of 2000, pg 14.)