Web Scraping Using PHP and jQuery

I was asked by a friend to write code that would scrape a DLP website’s content of letters to use in an academic study (the website’s copyright allows for the non-commercial use of the data).  I’d not tried this before and was excited by the challenge, especially considering I’m becoming more involved in “big data” studies and I need to understand how one might go about developing web scraping programs. I started with the programming languages I know best:  PHP & jQuery.  And yes, I know that there are better programming languages available to write code for webscraping.  I’ve used PERL, Python, JAVA, and other programming languages in the past, but I’m currently much more versed in PHP than anything else! If I had been unable to quickly build this in PHP, than of course I’d have turned to Python or PERL; but in the end I was able to write some code and it worked. I’m happy with the results and so was my friend.

First, I had to figure out what PHP had under the hood that would allow me to load URLs and retrieve information. I did some searching via Google and figured out the best option was to use the cURL library (http://php.net/manual/en/book.curl.php).  The cURL lib allows one to connect to a variety of servers and protocols and was perfect for my needs; don’t forget to check your PHP install to see if you have the cURL library installed and activated.  I did a quick search on cURL and PHP and came across http://www.digimantra.com/technology/php/get-data-from-a-url-using-curl-php/ where I found a custom function that I thought I could edit to suit my needs:

Next I needed a way to grab specific DOM elements from the pages being scraped; I needed to find a <span> tag that had a specific attribute containing a value that was both a function name and a URL.  I am very familiar with jQuery syntax and CSS3 syntax that allows one to find specific DOM elements using patterns.  Low and behold I discovered that someone had developed a PHP class to do similar things named “simplehtmldom” (http://sourceforge.net/projects/simplehtmldom/).  I downloaded simplehtmldom from sourceforge, read the documentation, and created code that would find my elements and return the URLs I needed.

Now I have the actual URLs from which I want to get  a copy of the data in an array.  I need to loop through the $links array and use cURL once again to get the data.  While I’m looping through the array I need to check to see if the URL is pointing to an HTML file or a PDF file (my only two options in this case).  If it is an HTML file, I use the get_data() function to grab the data and use PHP file commands to write/create a file in a local directory to store the data. If it’s a PDF, I need to use different cURL commands to grab the data and create a PDF file locally.

That’s it for the scraping engine!

Now we need to create a way pass a  start and end value (increments of 50 and maxes out at 4000) to the PHP scraping engine.  I know there are many ways to tackle this and I specifically considered executing the code from a terminal, in a CRON job, or from a browser.  I again went with my strengths and chose to use an AJAX call via jQuery.  I created another file and included the most recent jQuery engine.  I then created a recursive jQuery function that would make an AJAX POST call to the  PHP engine, pause for 5 seconds, and then do it again. The function accepts four parameters: url, start, increment, and end.

Put this all together and we have a basic web scraper that does a satisfactory job of iterating through search results and grabbing copies of HTML and PDF files and storing them locally.  I was excited to get it finished using my familiar PHP and jQuery languages and it was a nice exercise to think this problem through logically.  Again, I’m SURE there are better, more efficient ways of doing this… but I’m happy and my friend is happy.

Fun times.


Scientometrics, Scholarly Communication, and Big Data… oh my!

Wordle Image
The Digital Humanities and Humanities Computing: An Introduction by S Schreibman, R Siemens, & J Unsworth

I’ve started the 2012 fall semester with a new G.A. position working for Dr. Cassidy Sugmioto on a grant titled Cascades, Islands, or Streams? Time, Topic, and Scholarly Activities in Humanities and Social Science Research.  The grant was awarded through the NEH and the Office of Digital Humanities and was part of the Digging Into Data challenge. The official grant description reads:

This project will examine topic lifecycles across heterogeneous corpora, including not only scholarly and scientific literature, but also social networks, blogs, and other materials. While the growth of large-scale datasets has enabled examination within scientific datasets, there is little research that looks across datasets. The team will analyze the importance of various scholarly activities for creating, sustaining, and propelling new knowledge; compare and triangulate the results of topic analysis methods; and develop transparent and accessible tools. This work should identify which scholarly activities are indicative of emerging areas and identify datasets that should no longer be marginalized, but built into understandings and measurements of scholarship.

I’m extremely excited about this G.A. position! It will allow me to study, record, and understand communication, connections, and behavior in social network sites (SNS) using a different set of tools and theories and it will allow me to make use of my semester AI’ing with Dr. John Walsh on his S657 Digital Humanities course and my past five years of experience working in the digital humanities (DH) realm and developing/designing DH websites and tools for the Chymistry of Isaac Newton project, TILE, and other projects.

It’s an ideal opportunity and I’m so lucky that I asked to audit Dr. Sugimoto’s Ph.D. version of her Scholarly Communication course.  I was interested in the scholarly communication beforehand after examining how Erving Goffman was cited in a subset of information science (IS) literature (a large portion of the discourse simply cited him ceremoniously). I became quite involved with and interested in the subject of scholarly communication as the course progressed. The course and discussion opened my eyes to vast possibilities outside the simple “citation count” and I became quite interested in the discourse. Later Dr. Sugimoto approached me with the possibility of working together and I jumped at the opportunity.

I came to the project with both excitement and fear, fear primarily because I felt a bit out of my comfort zone as the scholarly communication discourse was still relatively new to me. The other faculty and students working on the project have been fantastic and I’m positive I’ll benefit in a variety of ways from the experience. The fist part of my assistantship will include working with a small group of Ph.D. and masters students set with the task of examining network characteristics of the DH community across a variety of sources including various social network tools/sites, journals, books, and listservs, to name just a few. Some in the DH community have visualized and discussed characteristics of the DH populace (Cleo, Melissa Terras, and Alex Reid, to list just a few) and our group hopes to add to this picture by examining other sources for characteristics common to the DH community.


Integrating Social Capital, Impression Management, and Privacy

At this time, my dissertation looks to involve the integration of theories and frameworks related to social capital, impression management, and privacy. Impression management is the easiest theory for me to deal with because I have a strong attraction to Goffman’s dramaturgical framework. While I know that his is not the only framework or theory relating to impression management, at this point in time, it is the framework to which I’m attracted. The big three of the social capital world, based on my own readings, include Bourdieu, Coleman, and Putnam. I tend to lean toward Coleman’s definition of social capital only because it is less egocentric; Bourdieu’s definition and explanation is extremely egocentric and Putnam’s views are at a macro level and include large organizations and groups. Privacy research is a grab bag of ideas and theories composed of a variety of definitions and understandings.

I’m attempting to integrate these theories to explain behaviors in the domain of social networking sites (SNS). SNSs are an extremely hot topic in academia and in the popular press. Facebook is pushing 700 million users and Google is now entering the realm with Google+. The SNS environment is an extremely interesting domain to investigate because

    1. SNS such as Facebook have so may users
    2. The computer-mediated environment presents us with different challenges during interactions than face-to-face interaction
    3. It is a relatively new domain of study and has continued to show growth over the past 7+ years
    4. Popular media has made blanket statements regarding interaction, privacy, and safety within the SNS environment that academicians have set out to examine
    5. Web 2.0 technologies have allowed for a variety of media to be transmitted through the SNS frameworks that affect self-presentation, social capital maintenance, and privacy

While the SNS domain is ripe for investigation, I also feel that a successful integration of these theories will be allow us to examine a variety of computer-mediated environments.

So far I’ve explained that I want to integrate impression management, social capital, and privacy theories and frameworks to investigate computer-mediated environments such as SNS. I’ve left out a crucial component… WHAT will I be investigating? This is the ultimate question and what is giving me the hardest time at the moment.

“Indiana University project releases more of Sir Isaac Newton’s alchemy manuscripts” from IUB Newsroom

Story from http://newsinfo.iu.edu/web/page/normal/19929.html

"Star regulus of antimony" was produced by the Chymistry of Isaac Newton project following directions written by Newton.
"Star regulus of antimony" was produced by the Chymistry of Isaac Newton project following directions written by Newton.

The Chymistry of Isaac Newton project at Indiana University Bloomington has released digital editions of 30 previously unedited manuscripts written around 300 years ago by the great British scientist Sir Isaac Newton, the founder of modern physics.

The project, devoted to the editing and exposition of Newton’s work involving alchemy, the dream of transmuting base metals into gold, is directed by William R. Newman, Ruth N. Halls Professor of History and Philosophy of Science in the IU College of Arts and Sciences.

Look for a new Newton site design shortly!

Publicity on Newton Project, Moving on to TILE

SLIS News Story: http://www.slis.indiana.edu/news/story.php?story_id=2186

We have some new publicity on the Chymistry of Isaac Newton project. IU did a press release on the launch of new manuscripts, new website design, and new features… and I get some credit for redesigning the website. I’ve had a really great time working on the Newton Project with Dr. John A Walsh, Dr. William Newman, Wally Hooper, and the rest of the crew. We’ve had some good times. Now I’m moving on to the TILE project (Text-Image Linking Environment) and I’ll be working with Doug Reside, Dot Porter, Melissa Terras, and Dr. John A Walsh.

TILE Project Release: http://www.slis.indiana.edu/news/story.php?story_id=1985

Social Capital

(excerpt from working paper)

From FLICKR @napalm nikki
from FLICKR @napalm nikki's photos

Social capital is a concept that was born from economics and was defined in sociology. Simply put, social capital is a source of power that humans accrue through connections in a social network. Humans are social creatures and thus need to be connected to others in close-knit groups; these connections are two-way investments that are maintained through reciprocal support and investment. Social capital has become “one of the most popular exports from sociological theory into everyday language” (Portes, 1998, p. 2). Bourdieu (1985/2001) was the first in sociology to distinguish social capital as a type of capital in social relations; Bourdieu’s three types of capital are economic capital, cultural capital, and social capital.

Bourdieu defined social capital as “the aggregate of the actual or potential resources that are linked to possession of a durable network of more or less institutionalized relationships of mutual acquaintance and recognition – or in other words, to membership in a group – which provides each of its members with the backing of the collectivity-owned capital, a ‘credential’ that entitles them to credit in the various senses of the word” (Bourdieu, 1985/2001, p. 103). Social capital is not a form of currency in the sense that it is something that can be held and passed from hand to hand as material object, it is a type of value found only through social interaction and is built upon by social connections and affiliations, a commodity traded through trust and appreciation, through expectation of return and investment. Bourdieu goes on to state that the building of a social network presupposes a return of investment, “… the network of relationships is the product of investment strategies, individual or collective, consciously or unconsciously aimed at establishing or reproducing social relationships that are directly usable in the short or long term, that is, at transforming contingent relations, such as those of neighborhood, the workplace, or even kinship, into relationships that are at once necessary and elective, implying durable obligations subjectively felt (feelings of gratitude, respect, friendship, etc.) or institutionally guaranteed (rights)” (Bourdieu, 1985/2001, p. 103).

Social capital can be thought of as a resource derived from social connections with a network of individuals: actors in networks establish and maintain relationships with other actors in the hope that they may benefit in some way from these relationships. The relationships can be tight or they can be weak and this measure can have an impact on the return that an actor derives from either type. These social relationships can result in emotional support, the exchange of information, or mobilization toward a common goal.
Social capital is an extremely popular concept because it simplifies and focuses attention on two components: “First, the concept focuses attention on the positive consequences of sociability while putting aside its less attractive features. Second, it places those positive consequences in the framework of a broader discussion of capital and calls attention to how such non-monetary forms can be important sources of power and influence, like the size of one’s stock holdings or bank account” (Portes, 1998, p. 2). Bourdieu’s treatment of social capital is purely instrumental (ibid, p.3), breaking down the concept into a cost/benefit analysis in which he focuses on the advantages of group participation and the creation of network connections for increasing this advantage. Many other definitions in multiple fields have arisen since Bourdieu’s definition of social capital (Adler & Kwon, 2002).

Because of the variety of definitions (see Adler & Kwon, 2002, and Portes, 1998 for summaries), social capital could be considered a “contentious and slippery term” (Williams, 2006, para 4). Adler & Kwon (2002) categorize 20 social capital definitions into three categories: 1) definitions focusing on internal ties, 2) definitions focusing on external ties, and 3) definitions that focus on both types of ties. Those definitions of social capital focusing on external ties, also called “bridging” forms of social capital, focus primarily on the power of connections between actors in the network. The definitions focusing on internal ties, also called “bonding” forms of social capital, focus primarily on the benefits associated within belonging to a specific group or cluster of actors. The third group does not differentiate between internal or external types of social capital. In response to the argument that social capital isn’t really a resource and should therefore not be called “capital”, Adler & Kwon (2002) justify the term with seven rationale: 1) social capital is a long-lived asset which can be invested in; 2) social capital can be used toward multiple goals and can be exchanged for other types of capital; 3) social capital can substitute for other types of capital; 4) social capital requires maintenance; 5) social capital can be community owned; 6) social capital’s value is not in the individual, it is in the relationship(s); and 7) social capital investment is not easily quantifiable.

Another definition that is prominently cited in the literature is the definition of social capital put forth by Coleman (1988, 1990). Much as Bourdieu before him, Coleman breaks capital into three types: 1) physical capital, 2) human capital, and 3) social capital. Coleman defines social capital by its function stating that “[i]t is not a single entity, but a variety of different entities having two characteristics’ in common: They all consist of some aspect of social structure, and they facilitate certain actions of individuals who are within the structure” (Coleman, 1990, p. 302). Coleman’s definition differs from Bourdieu’s definition in that Coleman’s definition is much more vague and leaves room for multiple interpretations. Portes (1998) notes that Coleman’s vague definition “opened the way for relabeling a number of different and even contradictory processes as social capital” (p. 5). For Coleman, investigation of social capital must include three inquiries: 1) actors cashing in on social capital; 2) actors or groups providing the social capital; and 3) the resources used for social capital. Coleman’s definition was much more influential in American than Bourdieu’s and introduced the concept to American sociology (Portes, 1998).
In addition to Bourdieu and Coleman, Putnam (1995, 2000) added another definition to the growing list of social capital definitions when he released a book entitle Bowling Alone. This book characterized the social capital of American society as declining and garnished much attention from the general public. Putnam defined social capital as “features of social organization such as networks, norms, and social trust that facilitate coordination and cooperation for mutual benefit” (Putnam, 1995, p. 67). Putnam’s work expanded the focus from actors’ face-to-face interactions in social networks to internal ties with a society (e.g. America). Putnam argued that social capital would be valuable in fighting many disorders prevalent in today’s society. One of Putnam’s main contributions was the introduction of the distinction between “bridging” and “bonding” social capital. Bridging social capital is characterized by connections between actors in a social network that are used mainly for gathering new information, not emotional support. Bonding social capital, on the other hand, are connections made between actors in a close-knit environment (e.g. family, close friends) and are used for emotional support.

In addition to the concept of social capital, Granovetter (1973) and Lin, et al., (1981) are often cited in the literature for their concepts “strength of weak ties” and “strength of strong ties” accordingly. These concepts refer to the power of the connections between actors in dense social networks. Granovetter, examining employment referral, pointed to the power of indirect connections outside of an actors close social connections in social networks; his idea was interesting because it diverged from the commonsense notion that close connections, e.g. family connections, would be the most effective in a job search. Granovetter defined the strength of a tie as “a (probably linear) combination of the amount of time, the emotional intensity, the intimacy (mutual confiding), and the reciprocal services which characterize the tie” (Granovetter, 1973, p. 1361).
Lin, et al., (1981) focus their investigation on social resources that an actor has within a social network; they define social resources as “the wealth, status, power as well as social ties of those persons who are directly or indirectly linked to the individual” (ibid, p. 395). Lin, et al., like Granovetter, were interested in employment strategies and examined the relationship between a job seeker’s social network ties and the status of the job obtained. Lin, et al., found that as job status increased, the effectiveness of weak ties decreases whereas the effectiveness of strong ties increased. As might be evident, Granovetter’s notion of weak ties is very similar to Putnam’s idea of bridging social capital while Lin, et al., notion of strong ties is similar to Putnam’s idea of bonding social capital.

Outside the arena of information communication technologies (ICT), social capital has been used to study youth behavior problems, families, schooling, public health, education, political action, community, and organizational issues such as job and career success, innovation, supplier relations, to name a few (Adler & Kwon, 2002). As noted by the extent to which social capital has been used, the concept is in a sense a catchall term that captures aspects of social interaction that have been studied through the lens of other concepts.

References Used in the Larger Paper

Adler, P.S., & Kwon, S. (2002). Social capital: Prospects for a new concept. Academy
of Management Preview, 27(1), 17-40.

Bourdieu, P. (1985). The forms of capital. In J. G. Richardson (Ed.), Handbook of
theory and research for the sociology of education, 241-258. New York: Greenwood.

Coleman, J. (1988). Social capital in the creation of human capital. American
Journal of Sociology, 94, 95-120.

Coleman, J. (1990). Foundations of Social Theory. Cambridge, MA: Harvard
University Press.

Granovetter, M.S. (1973). The strength of weak ties. American Journal of
Sociology, 78 (6), 1360-1380.

Lin, N., Ensel, W.M., & Vaughn, J.C. (1981). Social resources and strength of ties:
Structural factors in occupational status attainment. American Sociological Review, 46(4), 393-405.

Portes, A. (1998). Social capital: Its origins and applications in modern Sociology.
Annual Review of Sociology, 24, 1-24. Retrieved from http://www.jstor.org/stable/223472 on Jan. 03, 2010.

Putnam, R.D. (1995). Bowling alone: America’s declining social capital. Journal of
Democracy, 6(1), 65-78. Retreived Feb 5, 2010 from http://muse.jhu.edu/journals/journal_of_democracy/v006/6.1putnam.html

Putnam, Robert D. (2000). Bowling Alone: The Collapse and Revival of American
Community. New York: Simon & Schuster.

Williams, D. (2006). On and off the ‘net: Scales for social capital in an online era.
Journal of Computer-Mediated Communication, 11(2), article 11. Retrieved from http://jcmc.indiana.edu/vol11/issue2/williams.html

Heidegger and Schön in Design Discourse

NOTE:  This post is a brief summary of a larger paper in the works…

In an attempt to establish a personal, reflective, and examined intellectual position in relation to design as a professional process of inquiry, thought, and action, I’ve been writing on Heidegger and Schön in design discourse. The position taken here is derived from personal involvement in the areas of human-computer interaction (HCI) and design, field experience as a web professional, and close reflection on discourse from multiple disciplines. Specifically, I have tried to compare and contrast Donald Schön‘s ideas of reflective practice with Martin Heidegger’s concept of circumspection. I am taking an intellectual position toward the practical use of both concepts in future design discourse.

Design discourse has moved from viewing the world as a perfect rationality, to viewing the world as a bounded rationality, to most recently viewing the world as an expandable rationality (Hatchel, 2001). What this means for design theory is a shift in focus from clear, definable problems to more real-world, contextually vague problems. This shift has brought about a new understanding of design by introducing more risk and unpredictability to design understanding. This is in direct contrast to the field of science where the reduction of risk and vagueness is sought. This risk and unpredictability has led those in the field of design to think more about their relationship with the situation, the client, the design of the particular, and how one learns to design. For Donald Schön (1987), relationships like these led to a search for how individuals actually learn and tackle problems they are faced with in context.

Donald  Schön (1987), in his examination of learning, introduced the concept of reflective practice. Reflective practice introduces the concepts of reflection-in-action and reflection-on-action. ­In the field of design, reflection-in-action is based on the idea that a designer should reflect upon each action while engaged in design activity. This leads to an internal dialogue in which the designer constantly investigates why actions were made by separating out objects for investigation, while trying to remain within the overall scope of the design situation. Reflection-on-action, as the name implies, indicates that the designer should think about the design process after the fact. These two aspects of reflective practice focus on expectations and previous knowledge brought to the process by the designer. This practice requires the designer to take objects out of situation in order to compare expectations brought to the process with actual results. Smith noted that reflection-in-action “is sometimes described as ‘thinking on our feet’. It involves looking to our experiences, connecting with our feelings, and attending to our theories in use” (Smith, 2001). The important point of Schön’s reflective practice model I would like to focus on is the idea that in order to reflect on an object, knowledge must be present beforehand in order to facilitate the contemplation required to reflect. As noted by Nielsen (2007), this idea of reflection-in-action is similar to Heidegger’s concept of investigating a tool in the ‘ready-to-hand’ position. When an object, or equipment, is used it tends to fade from conscious thought. However when there is a breakdown or expectations are not met, the equipment becomes our focus of thought. Nielsen argues that Schön’s reflective practice model focuses specifically on this occurrence. Before this can occur, Heidegger (1962) argues that one must allow the equipment to be in relation to the world in order to truly grasp the object as it is. To learn about the object, Heidegger argues that one must first look around (circumspection) and take in the world where the equipment presides before learning can occur.

In Being and Time and The Question Concerning Technology, Heidegger (1962, 1977) asks us to reassess the Western philosophical ideal that a person understands the world through methods of science (Dreyfus, 1991). Heidegger wants us to recognize this fault and shift our focus to a person’s actual activities within the world (being-in the world).  Winograd & Flores (1987) were among the first to call on the phenomenology of Heidegger within the field of HCI. They focused on Heidegger’s view of language and thought in its relation to technology. They hoped to utilize Heideggerian philosophy to improve upon our understanding of interpretation in the contexts of making the implicit become explicit, attacking breakdowns from a practical perspective instead of a theoretical perspective, and stressing that meaning is a social derivation. Since Winograd & Flores, other researchers in both HCI and design have used Heideggarian concepts to explain the phenomenon of interaction through the examination of context (Chalmers, 2004), familiarity (Van de Walle, et.al., 2003), interactive art (Coffin, 2008), artificial intelligence (Agre, 1997), tangible computing (Dourish, 2001), situated action (Suchman, 2007), transparency (Janlert & Stolterman, 1997), and philosophy of technology (Verbeek, 2005), to name just a few. The primary focus for HCI and design researchers has been on the Heideggerian ideas of equipment that moves between the scenarios of presenting as ready-to-hand and present-to-hand.

While these ideas are sometimes very helpful and allow for discussion of interaction and design in new ways, Heidegger’s concept of circumspection is of particular interest in the context of this paper. According to Heidegger (1962), as we move through world scenarios the equipment we encounter and the actions they perform present themselves to us at “varying degrees of explicitness and with a varying circumspective penetration” (ibid, p. 71).  We are constantly performing a dance with equipment and actions phasing between the states of ready-to-hand and present-at-hand. Heidegger noted that through circumspection, one first sees the relationships between equipment before one can see the equipment as things as themselves. By allowing the equipment of interest to be in relation to the situation, only then can people orient themselves to the objects. This process of orienting to an object is practical in nature, not theoretical. Through circumspection we allow the fundamental nature of the object to reveal itself in context, which can include the revealing of rules, purpose, and inspiration that add meaning to the object. In designerly terms, when presented with a design situation the designer must bring together dissimilar equipment in a specific context to allow the individual objects to reveal themselves as intelligible objects that can be understood. It is important to note that through this uncovering of equipment’s being, Heidegger believed there is also a covering; meaning that through uncovering of rules, purpose, and inspiration there is also a covering of rules, purpose, and inspiration that don’t reveal themselves in context. This covering occurs by the very nature of uncovering; for as one assigns ideas and language to an object in order to think about it in a particular situation, one inevitably pushes characteristics of the object away that aren’t relevant to the situation and therefore covers those aspects. Heidegger argued, therefore, that although circumspection is an important component of allowing equipment to be, one can never truly know equipment.

If Schön’s reflective practice model primarily focuses on learning when an object is ready-to-hand, then we must also consider the use of Heidegger’s circumspection for learning as it focuses on learning when an object or equipment is present-to-hand.  While much of Heideggerian philosophy is difficult to comprehend and many of the terms used are specifically German, Heidegger’s notion of circumspection should be considered when discussing the process of learning.  This is particularly true when discussing learning in designerly ways.

In design terms Schön wants the designer to reflect-in-action and reflect-on-action. To do so, the designer is required to understand the context and possess an ability to consciously remove a process, thing, or idea that typically didn’t meet some expectation from a situation and understand what variables led to the particular result. This requires the designer to bring an understanding of the situation, process, or thing to bear on the reflective practice. This understanding is built on inquiry and language, both of which are influenced by prior knowledge. In Schön’s notion of learning, a designer must bring with him or her previous knowledge and the capacity to reflect on a thing as it is happening in context to better understand the thing itself. This prior knowledge will influence the designer’s reflection and thereby influence the situation as a whole.

Heidegger believed that through the uncovering of things, there was also a covering. This is meant to say that by defining and identifying a thing, process, or situation, one also thereby hides aspects of the thing, process, or situation that are not considered as part of the learned definition. Therefore, it would be misleading for a designer to merely reflect on design because the reflection is based on knowledge that has led to the covering of the thing, process, or situation. Another way to view the problem is through Heidegger’s idea of circumspection. As noted earlier, circumspection is the process of looking around, absorbing the situation, and allowing the thing or process to be in relationship to other things or processes in situ. By simply looking around, the designer can understand the object, process, or situation as they present themselves as things that are useful for something. This idea points to the process of learning as that which is revealed to us through use. In Heidegger’s idea of learning, what a designer knows is related to what the designer does and the context in which he or she does it; this knowledge stems not from theories or definitions, but from the way the designer interacts with the world. Heidegger stresses that to know, one must let equipment be in relation to the context where one encounters the equipment.  Knowing is embedded in the use of the equipment and can be discovered by observing relationships, context, and social practice. Schön stresses to know, one must reflect and learn by pulling objects out of context in order to understand why they may or may not have met expectations. These ideas could be construed as two sides of the same coin; Heidegger’s circumspection and Schön’s reflective practice both address opportunities to learn about equipment in context.

The position taken here is derived from HCI and design literature and field experience as a professional designer. This argument has highlighted some key aspects of Donald Schön’s reflective practice and Martin Heidegger’s circumspection and compared and contrasted both concepts to present an argument for both to be considered in future design inquiry.  As design has evolved, numerous theories have been borrowed from other fields in order to help those studying design and those practicing design establish a dialog that can help shed light on ideas and frameworks used to describe the design process.

By focusing on Schön’s idea of reflective practice and Heidegger’s description of circumspection, I hope to continue this dialog between practice and learning by introducing concepts from education and philosophy which can contribute to the designerly way of thinking. Although much has been written on Schön’s reflective practice, Heidegger’s notion of circumspection has largely been ignored. Heidegger has been used in HCI quite often to discuss present-at-hand and ready-to-hand objects in terms of interaction, but his concept of ‘looking around’ to learn has been largely ignored. While Nielsen (2007) has called for similar attention to Heidegger’s concept of circumspection in the field of learning, design has yet to follow suit.  By using both concepts to examine design situations, design academicians and design practitioners can lean on similar vocabulary to create a dialog between discourse and practice that can further design understanding.

References Used in the Larger Paper

Agre, P. (1997). Computation and human experience. Cambridge, UK: Cambridge University Press.

Bødker, S. (2006). When second wave HCI meets third wave challenges [Keynote]. In A. I. Mørch, K. Morgan, T. Bratteteig, G. Ghosh and D. Svanæs (Eds.), Proceedings of the Fourth Nordic Conference on Human-Computer Interaction (NordiCHI2006), 1-8.

Chalmers, M. (2004). A historical view of context. Computer supported cooperative work (13)3-4, 223-247.

Coffin, J. (2008). Interactive art, HCI and hermeneutic interpretation. Presented at the Alt.Chi. Session at the SIGCHI Conference on art.science.balance, Florence, Italy, April 5 – 10, 2008.

Cross, N. (1982). Designerly Ways of knowing. Design Studies, 3(4), 221–227.

Cross, N. (2001). Designerly ways of knowing: Design discipline versus design science. Design Studies, 17(3), 49-55.

Dourish, P. (2001) “Being-in-the-world”: Embodied interaction. In Where the action is. The foundations of embodied interaction. Cambridge, MA: MIT Press.

Dreyfus, H.L. (2007). Why Heideggerian AI failed and how fixing it would require making it more Heideggerian. Philosophical Psychology, 20(2), 247–268.

Friedman, K. (2003). Theory construction in design research: Criteria, approaches, and methods. In Design Studies, 24, 507-522.

Harrison, S., Tatar, D. and Sengers, P. (2007) The three paradigms of HCI. Presented at the Alt.Chi. Session at the SIGCHI Conference on Human Factors in Computing Systems San Jose, California, USA, April 28 – May 03, 2007 CHI ’07.

Hatchuel, A. (2001). Towards design theory and expandable rationality: The unfinished program of Herbert Simon. Journal of Management and Governance, 5(3-4), 260-273.

Heidegger, M. (1962). Being and time. (J.Macquarrie & E.Robinson, Trans.). New York: Harper & Row. (Original work published 1927).

Heidegger, M. (1977). The question concerning technology. (W. Lovitt, Trans.). New York: Garland Publishing. (Original work published 1952).

Nelson, H. & Stolterman, E. (2003). The design way – Intentional change in an unpredictable world. Educational Technology Publications.

Nielsen, K. (2007). Aspects of a practical understanding: Heidegger at the workplace. Scandinavian Journal of Educational Research, 51(5), 455–470.

Schön, D. (1987). Educating the reflective practitioner [PDF document]. Retrieved from http://educ.queensu.ca/~russellt/howteach/schon87.html

Smith, M. K. (2001, July). Donald Schon (Schön): Learning, reflection and change. The Encyclopaedia of Informal Education [Online] Retrieved Sept. 19, 2009, from http://www.infed.org/thinkers/et-schon.htm

Suchman, L. (2007). Human-machine reconfigurations : Plans and situated actions, (2nd. ed.). New York, NY: Cambridge University Press.

Van der Wall, G., Turner, P., & Davenport, E. (2003). A study of familiarity. In (M. Rauterberg, et.al., eds.) Human-Computer Interaction – INTERACT’03, 463-470. IOS Press.

Verbeek, P. (2005). What things do: Philosophical reflections on technology, agency, and design. University Park, PA: Pennsylvania State University Press.

Winograd, T. & Flores, F. (1987). Understanding and being. In Understanding computers and cognition: A new foundation for design. Reading, MA: Addison-Wesley.

Chymistry of Isaac Newton Annotation Tool Development

Firefox Add-On Development Center Logo

For the past few years I’ve been working on and off toward the development of a web-based text and image annotation tool for the Chymistry of Isaac Newton Project; I’ve been a member of the team since Fall 2007.   If you have any experience with web programming, you will know that it is almost impossible to successfully navigate the waters of browsers, operating systems, javascript engines, client requirements, accessibility standards, usability issues, mobile development, etc.  (the list continues to grow as new technologies saddle up to the Internet and demand seamless service).

The development has been long and arduous with many pitfalls, yet somehow I’m still working and making progress toward our goals…  I’ve learned a few valuable lessons, including my personal shortcomings as a designer, programmer, manager, student, friend, and husband.

I’m using XUL, jQuery, PHP, and MySQL for the beta development.  I chose jQuery mostly because our Digital Library Team here at Indiana University have some experience with this javascript library and are using it on a few other projects.  I would consider myself an expert with PHP, so that was a no-brainer.  I am at an almost expert level with both MySQL and PostgreSQL, so either database backend would suffice.   The XUL development choice comes from the fact that we are prototyping the annotation tool as a Firefox extension.  This part of the project has been the most frustrating and annoying, yet I’ve managed to get something working and we are moving forward now at a pretty good pace.  Now if you go looking for Firefox extension tutorials, you’ll be in luck because Mozilla finally got their act together and created a developer hub!  Finally!  This is very frustrating because it has come about 2 years too late for me…

Now I am going to start posting pointers from my own experience with this development life cycle.  I will post links to some tutorials, some hacks, a few scripts, and other materials that helped me along the way.  But maybe you won’t need them with this fancy, new dev center!