Cognitive NLP Research

Using AI Language Model Principles to Analyse Design Protocols

Morteza is "ready to rise to the formidable challenge of automating the coding and linking stages of protocol analysis and linkograph construction, which they rightly claim requires a lot of labor when done manually." Gabriella Goldschmidt, Linkography: Unfolding the Design Process. P141

Coding and linking concepts in design protocols is one of the most time-consuming and reseource-demanding stages of doing verbal protocol studies. A main challenge here is classifying the concepts based on the context and intention. One would think that, like in other domains, a problem is always a problem, and a solution is always a solution. In medicine, for example, one would rarely expect to see a disease as a medicine, or a medicine as a symptom. In design, however, the same concept could be considered a problem or a solution depending on the context. For instance, while designing a chair for exterior use, an industrial designer might suggest plastic as the material that could withstand the weather. Here, \textit{being plastic} is a solution in response to the need for the chair being used outside. Now imagine if in another project, the brief was asking for the chair to be made out of plastic. This time, \textit{being plastic} brings a collection of challenges and issues to the design space, i.e., it is part of the problem. This characteristic of design concepts and their dependency on the context and intention poses great challenge to any study, attempting to automatise the coding process. The recent rise of context-aware language transformers such as chatGPT, has created the possibility of doing so. However, it is not clear if the lingual context, i.e., the co-existence of words as used by context-aware language models is enough to identify problem and solution concepts.


Problem-Solution Index

Different studies have noted that designers shift their focus frequently between two ’spaces’ of problem and solution (Asimow, 1962; Lawson, 2005a; Scho ̈n, 1995). The designers’ focus on one space over the other is thought to be dependent upon different variables, including their experience (Cross, 2004; Restrepo and Christiaans, 2004), the type of design problem (Goel, 1995) and the design method they use. Dynamic problem-solution (PS) index is used to study the shifts in designer focus on problem and solution spaces.

For the purposes of this thesis, the dynamic PS index may prove useful when analysing problem-solving in customisation protocols. Customers might not iden- tify as many problem issues (e.g., requirements) about a product as designers do. Thus, simply counting the FBS design issues will not be enough when comparing customisation with designing. Calculating the ratio of the formulated problems to the generated solutions will create a better view of the customisation as a design problem-solving activity.

When calculating the dynamic PS index, the sum of requirement, function and expected behaviour issues is calculated as the problem index for each window position. Similarly, the sum of the structure and behaviours of structure issues is counted as the solution index of each window. Thus, the P-S index of the window is:

To facilitate a comparison of multiple protocols, calculation of fractioned PS indices (e.g., 1/10 fractions) is suggested; thus, every protocol will be presented as a ten-point graph of PS index values (Jiang et al., 2012). For calculation of fractioned PS index, the results of the dynamic PS index may be used; but, the window length must be equal to the fraction of the protocol size, and the window must be shifted forward as much as its length at every step.

The dynamic PS index is a singular value to indicate the designer’s focus on problem and solution spaces across the design session. The value is normalised to the number of segments in the protocol\fraction and is comparable between different design protocols. Moreover, the PS index creates conceptual connections between design issues from FBS ontology and the idea of problem and solution spaces that is quite popular in design studies. This is an important connection which enables a comparison of the findings from protocol studies where the FBS coding scheme is used with the results of other non-FBS studies (Jiang et al., 2012).

Using Attention Matrix Approach to Analyse the Framing in Design



You need  Java Virtual Machine to run LiNKODER. Once Java is installed, you should be able to run LiNKODER regardless of your operating system or it's version.