Comments

Commented on Kristine Harveaux-Lundeen's blog and Letisha Goerner's 2600
blog

Reading Notes November 17

Shreeves, S. L., Habing, T. O., Hagedorn, K., & Young, J. A. (2005). Current developments and future trends for the OAI protocol for metadata harvesting. Library Trends, 53(4), 576-589.

This text was not easy to understand, but here are some excerpts:

"The mission of the Open Archives Initiative (...) is to "develop and promote interoperability standards that aim to facilitate the efficient dissemination of content" (Open Archives Initiative, n.d. a). The Protocol for Metadata Harvesting, a tool developed through the OAI, facilitates interoperability between disparate and diverse collections of metadata through a relatively simple protocol based on common standards (XML, HTTP, and Dublin Core)."

"The OAI protocol requires that data providers expose metadata in at least unqualified Dublin Core; however, the use of other metadata schmas is possible and encouraged. The protocol can provide access to parts of the "invisible Web" that are not easily accessible to search engines
(such as resources within databases) (Sherman & Price, 2003) and can provide ways for communities of interest to aggregate resources from geographically diffuse collections."

-- The data is searchable and browsable without any manual cataloging of the various OAI repositories.

-- I could not pull up the text on how search engines work today, and will continue the reading tomorrow, once I get access to another computer.

White Paper: The Deep Web: Surfacing Hidden Value
Michael K. Bergman

Journal of Electronic Publishing, vol. 7, no. 1, August, 2001
DOI: http://dx.doi.org/10.3998/3336451.0007.104


-- This article is very enlightening -- it covers a technology (DeepPlanet, which can search the deep web)

-Most of the Web's information is buried far down on dynamically generated sites, and standard search engines never find it-- it si about 500 times the size of the surface web

--Traditional search engines create their indices by spidering or crawling surface Web pages. To be discovered, the page must be static and linked to other pages. Traditional search engines can not "see" or retrieve content in the deep Web

--The deep Web is qualitatively different from the surface Web. Deep Web sources store their content in searchable databases that only produce results dynamically in response to a direct request.

-- The deep web is a very coveted commodity

-- Study by NEC research initiative that search engine sby google and northern light only crawl 16% of the web's content

--Search engines obtain their listings in two ways: Authors may submit their own Web pages, or the search engines "crawl" or "spider" documents by following one hypertext link to another. The latter returns the bulk of the listings. Crawlers work by recording every hypertext link in every page they index crawling.

-- The crawls used to be indiscriminate, but "the most recent generation of search engines (notably Google) have replaced the random link-following approach with directed crawling and indexing based on the "popularity" of pages. In this approach, documents more frequently cross-referenced than other documents are given priority both for crawling and in the presentation of results. This approach provides superior results when simple queries are issued, but exacerbates the tendency to overlook documents with few links."

__ the problem here: without a linkage from another Web document, a page will never be discovered.

-- They don't use the term invisible web -- it is not invisible, but rather unindexable

-- the article continues to describe the study in more detail

--it is impossible to completely index the deep content, but new technologies need to be developed to search the complete web

Searching must evolve to encompass the complete Web.

Muddiest Point November 10

I wasn't quite sure if The University of Pittsburgh had an institutional repository -- if it does, how can you access it?

Assignment 5: Koha

Here is the link to my Koha collection on managing oral history archives:

http://upitt04-staff.kwc.kohalibrary.com/cgi-bin/koha/virtualshelves/shelves.pl?viewshelf=39

Comments

Commented on Preserving Tim's blog and Letty's LIS 2600 blog.

Muddiest Point, Week 10

I don't have a muddiest point for last week, but I do have several for this week. Here is one:
The article by Bergholz on xml was very helpful. It was, however, somewhat dated (from 2000). What has happened since the article was published -- have xml schemas replaced dtds? How have the development of rdfs and doms advanced since 2000?

Reading Notes Week 10 (November 10)

Bergholz, XML Tutorial
The article by Andre Bergholz is the clearest of all assigned articles, and I understood the other articles better after reading it. I really liked that he included examples, and also used library catalog examples, so you can see how it is applied in the library environment. It is, however, somewhat dated (from 2000), so it would be nice to read an up-to-date version of this on xml.
-- makes clear that html is layout oriented, whereas xml is structure oriented
-- both sgml and html influenced the development of xml
-- xml is about meaningful annotation
-- syntactically, xml looks like html
-- well formed document begins with prologue and has one element, and additional number of instructions can be added
-- dtds define the structure of xml documents
-- dtd elements can be nonterminal or terminal
-- elements can have zero or more attributes
-- extensions to xml include namespaces and addressing and linking capabilities
-- namespaces and dtds do not work well together
-- in html, links only point to a document
-- html is one-way
- xml extends htmls linking capacities
-- extended links connect more than one document
-- there is also an extensible stylesheet language, two languages: a transformation language and a formatting language
-- dtds have disadvantages and xml schemas address these disadvantages
-- xml schemas is well formed xml
-- the goal is to replace dtds
-- development of rdfs and doms also affect xml

Ogbuji, A Survey of XML Standards
--The world of XML is growing, with a huge variety of standards and technologies that interact in complex ways.
-- Points out that it can be difficult for beginners to navigate the most important aspects of XML, and for users to keep track of new entries and changes in the space
--includes a list of resources, if we want to explore this in more detail

XML Schema Tutorial
--XML Schemas are successors of DDT (document type definition)
--XML Schemas secure data communications
--they are extensible
--the schema element is rot of every schema
--xml schemas define elemnst of xml files
--attributes are complex
--restrictions on attributes are facets
--a complex element contains other elements or attributes
Martin Bryan. Introducing the Extensible Markup Language (XML)

This article was very difficult to follow for a non-programmer. What I understood from it, however, was that xml defines the structure of a document, rather than its appearance (like html). Therefore, xml is ideal for databases, so it can pull different elements from different locations together. “By storing data in the clearly defined format provided by XML you can ensure that your data will be transferable to a wide range of hardware and software environments. New techniques in programming and processing data will not affect the logical structure of your document's message.”