3              Criteria for scientific study and design

Taeke de Jong, Theo van der Voordt


3.1      Study and research............................................... 1

3.2      Design related study.............................................. 1

3.3      A definition of science........................................... 4

3.4      Criteria for empirical study (research)................... 4

3.5      Differences between research and design.......... 6

3.6      Design as a science.............................................. 6

3.7      Criteria for design related study ex post............... 7

3.8      Criteria for design related study ex ante............... 8


Could a design be the product of scientific work to be compared with a scientific report? If so, under which conditions and when? The topic is eagerly discussed both within and outside of faculties of architecture. On the web-site of the Design Research Society (DRS) there is a lively debate on what a design study and a study by design really are and when a designer can also be designated a scientist.[1] These questions stood central during the 1996 EAAE Congress organised by the Delft Faculty of Architecture on the theme ‘Doctorates in Design + Architecture’.[2] In order to answer these questions we discuss first the terms ‘research’ and ‘study’ and the usual pre-requisites that must be met for study to be designated ‘scientific’. Next, similarities and differences between designing and studying are dealt with. Following that, we discuss the usual way in the scientific community of looking at the criteria for a design to be branded as a product of scientific study. For that purpose a summary is given of the requirements the Technical University in Delft associates with a scientific design. Subsequently a short comment is given as well as the rôle it played during the initiative leading to this handbook of design related study. Finally we give a specimen of criteria for evaluation of a scientific architectural design (ex post) and of a proposal for a design related study (ex ante).


3.1           Study and research

Study is a collective term for generating knowledge by thoroughly thinking through a problem, carrying out experiments and collecting, processing and analysing data. When the primary purpose of a study is to know more, it is termed a fundamental study. When a study is mainly focused on practical usefulness – e. g. making better buildings, or contributing to more effective and efficient building processes – one speaks of an ‘applied’ study. If a study is mainly aiming at making a new product – a prototype of a (industrial) product, a constructive solution or a building method – it is termed (product)development.

In a sense, each and every one of us deals one time or another with study. Even the student studying Ernst Neufert’s ‘Architect’s Data’ to ascertain how large a class-room should be, busies himself with study. In that case searching, re-searching and retrieving come to the fore.[3] For a study, generating new knowledge is typical. The contrasts are not always well-defined. Just looking up something may develop into a lot of retrieving and you might find yourself in a genuine study.


In scholarly circles the term ‘research’ is often employed, rather than the concept ‘study’. Designers also use both terms; e. g. the ‘Design Research Society’, and its ‘Design Research Newsletter’, the ‘Journal of Design Research’ and the magazine ‘Design Studies’. Another term is ‘inquiry’, as in the title of John Zeisel’s book ‘Inquiry by Design’.[4] Related terms are ‘survey’, ‘investigation’ and ‘examination’. This book is trying to conform itself to the British English distinction between study and research. Research is roughly the empirical form of study. The term ‘research’ originated some hundred years ago and is used a synonym for the older term ‘study’, especially in the USA. Britons tend to employ the term ‘study’ for looking for something that does not exist as yet, in a broad sense: such as for subjects from non-empirical branches of logic and mathematics, but also for studies of Rembrandt, designers and students.


3.2           Design related study

A design does not follow unequivocally and reproductably from a programme like a scientific prediction repeatable from its basic assumptions, ‘ceteris paribus’. In making a design, the preliminary investigation and its conclusion, the programme of requirements, direct the solution only partly. Even within the boundaries of a strict programme, unexpected and unpredictable alternatives are possible in design. Most design decisions about form, subsequent structure (set of necessary connections and separations to keep the form) and even subsequent function (freedom of unexpected use) must be made without empirical evidence. This is most explicit in building design. The choice of a final alternative is determined by the total context of the object to be designed. The programme of requirements reflects only a small part of that context. Location, market and designer (context of invention) belong to the broader present and future managerial, cultural, economical, technical, ecological, and mass-space-time context and perspective of the object. Context’ is different on different levels of scale and cannot be foreseen completely in the programme.


The number of imaginable alternatives for buildings, mostly with a long term multi-functional programme of (conflicting) demands, is uncon­ceivably large, subject to a combinatoric explosion (see page 208) of possible forms. Buildings and urban designs have a long period of use and are earthbound. So they have to function in a changing context that is unpredictable and not influenced by the programming authority, designer or user. From the viewpoint of durability they should be able to accommodate varying programmes and the daily changing aims of inhabitants and users. This quality of building design is called ‘robustness’. ‘Flexibility’ is only part of it. So, from all artefacts, buildings have the most context sensitive function for use, perception and market, not to be evaluated without that context and, therefore, hardly comparable to each other (sometimes even unique).


Even with a comparable programme of requirements, not only the diversity of solutions, but also the diversity of contexts or perspectives to function in, is very large. Consequently, the diversity of rational reasons (determined by context) to opt for a final alternative is even larger. So, building design research often has the character of an n=1 study (case study) with limited general value to other designs. Design research, based on more examples than one, is often ignored by designers, because on location many design relevant circumstances appear different from what the examined examples had in common. The principal often demands a unique design, ‘exploiting’ rare and distinctive qualities of context. The descriptive interpretation of context by researchers differs from the imaginative interpretation of designers that stresses possibilities rather than probabilities.


In a University of Technology, designs are made not only intuitively, but based upon study (design study) and documented, examined and evaluated (design research). Design research concerns determined objects within determined contexts. ‘Study by design’, in a broad sense, varies either the object (design study) or the context (typological research) or even both (study by design):


Figure 1 Types of design-related study


The terms from this matrix may be explained as follows:


Design research

Design research describes and analyses existing designs with a known context, often in the form of comparative study. For that reason it is  evaluating study ex post. Not only their function is involved, but also their form, structure and the way they were made, the design tools employed in each stage and the way in which they were applied: the making proper.


Typological research

Whenever the identical architectural form, structure, technique, function or concept is recognised in different contexts the notion of a ‘type’ is involved. A type only becomes a consistent model if it has been elaborated for evaluation by design in a context. A type is a design tool, not yet a model. The study of such types, their use in the making of designs (a special kind of models) is called typological research.


Design study

Making a design in a relatively well-known context of potential users, investors, available techniques, building materials, political, ecological and spatial restrictions, entails many stages of a type of study termed in this book ‘design study’. If, in the case of grand projects, parts of it are sub-contracted, the parlance is ‘study for the designing’ or ‘research driven design’.


Study by design

Characteristic for this type of study is generating knowledge and understanding by studying the effects of actively and systematically varying of both design solutions and their context.


Only if both context and object have been determined (design research ex post), pure empirical study may be largely depended upon ; although that should also be done with the eye – and sometimes the hand – of the designer. Empirical (historical) design study is sometimes calling for a design re-construction of the design or of the design process.[5]

For the other three guises of study the designing itself must play a crucial rôle, although an empirical component will remain present in the form of researching inventories, descriptions, programming or evaluating research. Also, for the broader context of these studies the designing study may be the object of design.

In the case of a type of study with a determined object or context, the typology and the design study (daily practice of the profession) a lot of experience has been attained. When both are variable (study by design), a way out may be found in inter-changing typological research and design-study. This way, now the object, then the context is varied. However, it can not be excluded that this study can also stand on its own legs without using both methods of study. The first signs are the studies of Vollers[6] and of Frieling[7]. Vollers’ point of departure is the means for design as they manifested themselves in the usage of Computer Aided Design, from where possible objects and contexts for application are getting shape. Frieling’s point of departure is a dynamic public weighing between projects on a small scale (objects) and perspectives on a grand scale (contexts), within the domain of coming to decisions for the Delta Metropolis. Graduation – when those who graduate are allowed to determine themselves context and object – has resulted in an archive of experiments, some more successful than others, exactly in the field of study by design.


Taken together, these modes of study are termed ‘design related study’. Because of the inter-action between designing and studying, the borderline between both is not always clear cut. Actually a gliding scale between art and science applies.


Figure 2 Domains according to Van der Voordt


In this diagram studying and designing both feature their own domain; while the two overlap. At that point one may imagine study activities without design and design without activities of study.


One may also maintain that all empirical study pre-supposes a designed hypothesis (possibly put to work by way of a model) and a toolbox of research, so that empirical study pre-supposes some kind of designing as well; for the model of the reality (hypothesis) to be checked against that reality and the toolbox enabling observing, checking and predicting must have been designed earlier themselves. Without these conditions study cannot be imagined. In this sense the telescope as well as several branches of mathematics have been designed for modern empirical astronomy; next they were a condition for it; and finally it pre-supposes them. In that way design is always pre-supposed in study and research.


Figure 3 Domains according to De Jong


If these pre-suppositions are forgotten one forgets as well that their reliability is always open for discussion.


Experimental empirical research can also produce unexpected possible futures, however, only because it pre-supposes the design of the experiment and its instruments. Bacon[8], cited by Kant[9] in his Preface (Praefatio), states that science has not to be concerned as opinion, but as work (‘…non Opinionem, sed Opus…’). Elsewhere Bacon states that nature has to be forced to answer the question of the scientist. The scientist has firstly to design the experiment in order to produce improbable events in some future. Kant states in his preface that scientists before Bacon understood that human reason only recognises what it produces itself by design (‘…das die Vernunft nur das einsieht, was sie selbst nach ihrem Entwurf hervorbringt.’).


Thomas Kuhn, who created the ‘paradigm’ concept, associates his description of scientific revolutions closely with making tools and instruments.[10] Van der Meer, the Dutch engineer who designed for the Geneva Cyclotron the type of improvement that caused the discovery of crucial new facts in nuclear physics, got the Nobel Prize, not for those facts, but for bringing them to light. When one regards mathematics as a tool box for working models, mathematics and models built with it are also design instruments in need of design. This happens during empirical study and is caused by it. All statistical checks came into existence this way. In its usual, more narrow sense, designing does not relate to models simulating probable futures, but possible futures; even if they are not likely. This narrower sense of ‘designing’ is emphasised in this book.


3.3           A definition of science

An important question is now: when may a study be termed ‘scientific’? And especially: when is design-related study scientific? In order to answer this question, one must first define what ‘scientific’ entails. Although several definitions are current, over and over again a number of properties are returning in almost all definitions. This leads to the following definition:


Science equals any collection of statements that features a reliable relationship to reality, a valid mutual relationship and a critical potential with regard to other statements in the same domain.


The term ‘reliable’ may not only relate to ‘true’ and ‘probable’ in empirical sciences, but also to ‘feasible’, ‘working’, and, therefore, ‘possible’ in technical sciences. In both cases ‘reliable’ pre-supposes ‘verifiable’, ‘documentable’ and by the same token a public domain and accessibility of sources and methods (see page 92).


The term ‘valid’ (see page 92) pre-supposes the validity with which lines of reasoning may be constructed out of propositions and propositions out of statements (logic, see Chapter 23 , page 189) or the completeness with which the context is taken into account in a proposal or demonstration. This last criterion is particularly important for technical sciences. Completeness not only concerns a larger, but also a smaller context of gaps filled with tacit suppositions. Incomplete knowledge is half truth: incomplete technology is failure. In empirical science the completeness may be partly covered by the 'ceteris paribus’ pre-supposition (“for so far the rest is equal”); in technical sciences this is seldom feasible. Completeness can never be reached entirely and is by the same token a relative concept, but a given proposal (like a design) may be more complete than another (like considering more contextual effects of the proposal).


The term ‘critical’ pre-supposes that it is possible to make a statement that refutes other (e.g. ‘popular’) statements and that it is also possible to refute the statement itself (see paragraph 3.4, open to criticism, able to criticise).

The term ‘domain’ relates to the collection of subjects from reality evoking a statement. The term ‘statement’ encompasses propositions about a probable reality as well as proposals for a possible reality. The term ‘reality’, therefore, has a wider meaning than empirical ‘existing reality’ (think of notions like realistic e.t.q.). The concept ‘potential with regard to’ is wider than the concept ‘connected to’.

Many other definitions are current.[11] We choose this one in order to give technical sciences a place next to empirical sciences (here denotated by ‘empiry’). Scientific study is the activity needed to arrive at scientific statements. Not all study is scientific. It is characteristic for scientific study that its results are reliable, valid, capable and open to criticism. In the case of empirical research these criteria may be further specified.


3.4           Criteria for empirical study (research)


Empirical reliability entails that repetition of a measurement under unchanged conditions renders identical results of measurement. A ‘face value’ assessment of the constructive quality of a building is less reliable than measuring its physical shortcomings. ‘internal reliability’ is the parlance if the same investigator judges a particular situation more then once and each time comes to the same conclusion.

External reliability’ means that different investigators judge the same situation with the same results. Only if reliable instruments of measurement are used can a sufficient degree of objectivity apply.


Reliability necessitates formulation of a criterion of objectivity. Those who study, or are engaged in carrying out a particular project study, should strive to keep personal opinions from influencing the study and refrain from making personal value judgements. Someone else should be able to get the same results when using the same method. The instruments of measurement are thus severely tested. In a study of notions hard to measure, like architectural quality, or importance of a specific intervention in urban design, complete objectivity can hardly be realised. A careful description of concepts and measuring instruments, additional independent measurements (e. g. repetitive measurements by different investigators) and intermediary verification of findings by third parties does increase the probability of objectivity, often alluded to as ‘inter-subjectivity’.


A second criterion derived from reliability is verifiability. In order to qualify as a scientific study the structure of the study, the collecting of data, analysis of the material and interpretation should be made comprehensible to outsiders: it should be clear how the investigator reached his conclusions. This enables scientific debate. What is more: it offers other investigators opportunity to repeat the study; in different times, at different places. Naturally the requirement of verifiability requires a clear presentation and publication of the study.

While interpreting the data of a study and drawing conclusions, it is almost impossible to exclude personal (pre-)suppositions. Therefore, it is recommended to separate in the report of the study as much as possible the factual study results from the interpretation and conclusions. This leaves the possibility open to reach different conclusions based on the same material.


The term ‘value free’ is closely related to objectivity. This entails that the end – scientifically founded knowledge – justifies the undertaking of study, even if its results would clash with prevailing norms and values, or if the study would work out negatively for segments of a community. These days, practically everyone agrees that study can not, and should not, be value-free. Norms and values are important while choosing the inquiries of the study and the application of the study results. This does not preclude that within the given context the reasoning should be valid.



A second primary criterion for scientific study is validity. Amongst others, it means that what is measured is what is reported to be measured. Does measurement of temperature and humidity entail comfort? Who studies the effect of the presence of a ‘major domus’ (caretaker) on the intensity of vandalism in an apartment building by way of asking his opinion on it runs the risk to study rather the legitimacy of his appointment than the real effect. Measurement of costs of repair before and after his appointment is a more valid instrument of measurement.


This leads to the derived criterion that the investigator should think about the way how he might find efficiently and effectively the answer to the study question: he is looking for a methodical way, allowing research. Detours to reach a conclusion should be avoided. They might result in mistakes in the reasoning difficult to verify. Yardsticks are efficiency (using not more measurements, means or pre-suppositions than is necessary) en effectiveness (the method should be the answer to the question of the study). This requires thorough analysis of the problem, an inventory of sources of information available, a clear and unequivocally formulated statement of the problem and the purpose of the study, and critical reflection on the most appropriate study methods.


Over the years an extensive methodology of research has been developed. Presently there is a large variety in methods and techniques of research. For a survey and reflection on advantages and disadvantages as well as considerations as to selection we refer to the professional literature.[12] A methodological approach, by the way, does not mean that each step to be taken may be thought out in advance. Often progressing insight manifests itself while new steps are developed during the study. Additionally, rather accidental finds occur, some of them inspired by creative ‘flashes’ of insight: serendipity.


Open to criticism, able to criticise

A study may be reliable and valid, and yet not assessable. An important criterion of demar­cation is Karl Popper’s ‘refutability’ (openness to criticism, possibility of negation, falsification) of study results.[13] At stake is the possibility of counter-examples undermining the general conclusion.

For a long time the opinion prevailed that it would suffice, if the study results could be checked by others (verification). However, certain statements have been formulated in such a way that they may be verified always: speaking for themselves and by themselves. A well-known example is: “Everything is tripartite”. When an investigator shows in refutation a safety match, the object is broken into three by way of verification. Statements of that type are adding nothing scientific to existing knowledge, according to Popper. In contrast an example of refutable knowledge is Einstein’s theory of relativity, stating that light is bent by mass. Only in the years following publication of the theory this could have been refuted, if at a solar eclipse it would show that a star behind the sun would appear precisely on the moment predicted by the calculations of its course. The star appeared a little earlier, verifying the theory of relativity. Essential in this is that at that moment the possibility existed that the theory was refuted. The proposition was risky


In adddition to verifiable and open to criticism by third parties, scientific statements should also be critical themselves. The falsification principle of Sir Karl Popper is not only a passive, but also an active one. This means that science is open to both confirming and refuting existing opinions and views, for the time being seen as hypotheses. By checking them empirically or in terms of logical consistency, these hypotheses can be unmasked with more certainty as true or false. The potential to get away from myths is an important characteristic of science. This brings the criterion of scientific relevance into view.


Scientific relevance

Scientific study should widen and deepen development of the scientific discipline. The renewal or deepening may comprise contributing to the development of theory (generation of new knowledge, refuting or amending existing views), new methods and techniques of study, policy instruments and product development. Study limited to inventory of data is widely disregarded as scientific study. Even if the criteria of a methodological approach: reliability (objectivity, verifiability) and validity are honoured, scientific relevance is low, if it cannot criticise any existing suppositions. By itself this has nothing to do with social relevance or ethical admissibility: the contribution to improvement of the quality of life. At a level of low scientific relevance this might be high.


3.5           Differences between research and design

In spite of kinship between research and design differences apply:


·        The primary product of research is general knowledge in the form of probability. In a more narrow sense it is also the description of existing reality or truth belonging to it. This know­ledge may, or may not be applied in (design) practice. The primary product of designing is the representation of a possibility; also if it is not a likely one. A design demonstrates what is possible and thus may become reality. Knowledge of what is probable is always incorporated in a design, often implicitly; e. g. that a brick can endure a well defined pressure.


·        Research deals mainly with analysis; with a design process the focus is on synthesis. Analysis (etymologically ‘loosening’) severs a phenomenon from circumstance (context, set of conditions) and components (reductions) that are different elsewhere, in order to retain what may be made comparable fit for study (operational). That enables (ceteris paribus) statements that may be generalised. Synthesis integrates diverging requirements and interests, but adds in passing also conditions leading to new consequences for use and experiencing. Continuously the design process offers new opportunity, not be described ex ante and often not in words.


·        Research strives towards development of knowledge that may apply in several contexts. Research deals mainly with reality and experiences (empiry). Thus, this research is empirically orientated and its thrust is towards probability. What the reality should be like may also be subject of research. From social goals and norms (points of departure) one reasons backwards to means for reaching them (normative study). The personal opinion of the researcher, however, is not allowed to play a rôle in the interpretation of the data of the study (objectivity). In order to restrict an explosion of possibilities caused by combinatorics, designing is almost by definition coloured by personal preferences (selective attention for empirical facts) of the designer (subjectivity). Designing may be normatively biased; a characteristic it shares with the arts.


·        Usually different methods and techniques are employed in research and design. For instance: research of the literature, polls, interviews, measurement of characteristics of a building, and experimenting are common study methods. Common design methods include usage of metaphors, adapting existing types, or application of design principles. Lynch[14] proposes for example design principles in order to create a ‘legible built environment.


3.6           Design as a science

Given what was suggested until now, it is obvious, that designing as an activity, and a design as a product of this activity can only pass muster as a science, if the usual criteria for scientific activity have been obeyed. In an Advice of the Working Group Criteria Designing Disciplines (December 1999) this position is taken.[15] If a (tentative) design applies for being branded as result of scientific activity, it should comply with general requirements put to the scientific approach, to wit: inter-subjectivity, reliability and verifiability in an empirical sense. ‘Inter-subjective’ was defined by that group as ‘interpreted by different people in the same way’. ‘Reliable’ means here that the design demonstrates under different circumstances determined ex ante, behaviours that also determined ex ante. Verifiable points to the description of the design in terms of the grounds on which the decisions have been taken, including validity and tenability. Additionally, ‘verifiable’ includes, that the design can be specified according to concrete situations and can be generalised to possible applications in different situations or contexts. The working group derived the following concrete criteria:


·          Novelty vis-à-vis state of the art of technique and originality

·          Design methodological approach with a subjectivity that is argued

·          Construction and materialisation in reality, if applicable

·          Evaluation of actual performance of the design, compared to the performance intended

·          Integration of design, development and study

·          Integration of designing on different levels of scale (vertical integration)

·          Integration of partial designs and aspects (horizontal integration)

·          A vision on future development of the domain – in terms of design, discipline and science – the programme deals with.


To judge a design on these criteria, its presentation should include a description giving attention to these aspects.


During the EAAE Congress, mentioned earlier, ‘Doctorates in design and architecture’, comparable criteria emerged. Many scientists and designers agree that a design as a produce of scientific work should be based on a transparent process that may be assessed; a logically valid argumentation and accessible source of documentation. Originality, validity, economical use of means, clarity as to the underlying values and openness vis-à-vis verification and refutation are widely accepted criteria.

Nevertheless these conventional criteria allow some remarks. They have been strongly suggested to non-designing, truth directed disciplines with a preference for general knowledge. This may be a consequence of the fact that a lot of design related study has been done by social scientists, organisation experts, historians and technicians, not by the designers themselves. There must be something left over concerning design itself.


Restrictions to reliability

For multi-functional facilities or facilities used during a very long period, leaving open more possibilities of usage than foreseen, causes the requirement of reliability to be discussed. Someone immediately sees these possibilities, someone else after some time. A great number of possibilities of use and freedom to choose between them restricts reliability. The value of a multi-functional design sometimes increases with the number of possibilities of use in different contexts (robustness). By the same token, a conflict between this robustness and the reliability of the assessment may exist when evaluating an architectural object. Mono-functional facilities like a public water closet on the other hand may be evaluated reliably up to a point.


Restrictions to validity

Usage of an architectural design is, then, even more context sensitive than, for instance, usage of a petrol engine and consequently difficult to generalise. What works in one spatial, ecological, technical, economical, cultural and political context needs not to work the same way elsewhere. While architectural designers are hired particularly for solving, in a unique way, problems connected to place and context in a dynamic and many-faceted society, the classic empirical scientific striving towards statements that can be generalised may be frustrated.


There is an important distinction between the modalities “to be” and “can”. Everyone senses the incorrectness of the statement “That is not so, therefore it cannot be done this way”. Between empirical and technical sciences there is an important difference in modality.[16] What is probable inter-subjectively is per definition possible, but what is possible is not always probable. Improbable possibilities are seldom inter-subjective, as long as they have not been demonstrated by realisation. Before demonstration just a belief applies (with the possibility of realisation).


Designing concentrates on discovering these improbable possibilities. This puts the criterion of inter-subjectivity into jeopardy. Even after realisation proving the possibility of spatial construction inter-subjectively, the use of the facilities built in its parts is in principle unpredictable, as long as one believes in the freedom to choose on behalf of users. The value of an architectural design is determined by the degree in which the design offers its user new possibilities to choose from. A home does not cause homing, it just makes homing possible. By the same token, design thinking is less focused on causality than on conditionality.[17]


Restrictions to evaluative potential

A final remark regards verifiability in the case of categories which are not to be compared. Each and every design features elements like usefulness, beauty and sturdiness, that cannot be compared.[18] Nevertheless, it is precisely the way in which these incomparable categories have been unified consistently within a specific context that determines the value of the design. Before the building can demonstrate its value on the market ex post, the validity of considerations between these principally incomparable categories and defending them can not be objectified. Even if a building proves its value this way, this does not ensure that the experiment will lead to the same result somewhere else. In addition, it often happens that context specific reference material is lacking against which a design before execution (ex ante) can be checked, when thorough evaluations of comparable cases after realisation (ex post) are absent.


3.7           Criteria for design related study ex post

A scientific design should not be required to meet the criterion that its result is probable, as is the case of a study carried out in an empirical context. This puts a number of scientific criteria mentioned into jeopardy. There is even no need to require that a design is desirable, while improbable innovations often may not be imagined before they are proposed in a design. This is a crucial function of scientific design. As long as one does not know what is possible, one cannot know what one wants.


However, one must require that realisation of the design in one context or another is possible. The question is whether it must be socially possible at the same time. What is socially not feasible at present may become so when the possibilities have been brought into light. Even the question whether a design is economically feasible at present is no scientific yardstick, although a perspective may be required within which realisation may become possible at a certain time. Associated with this one should not require that the design has also been developed in a goal-directed way based on a statement of problems and aim ex ante (programme of requirements or brief). Rather, paradoxically, this pre-supposes an imagination of the result ex ante (hypothesis). It may be an experimental study orientated to a means with uncertain functions as a result.[19]


It is in order to ask which criteria remain. There are less of them than in empirical study, but from a viewpoint of the requirement of completeness there are also more. A suitable and extensive survey has been given by Eindhoven Technical University.[20] We restrict ourselves here to a minimum based on experience with evaluating matriculation designs and designs in other educational projects.


The following general criteria for technical university design on the level of a dissertation could apply to all technical sciences:


A.     The scientific design should be understandable to others in the culture given so that it can be judged by them (to be expressed in a rich way) and, therewith, open to control, criticism and refutation.[21] The scientific design has been drawn up, documented and discussed by the designer with a clarity sufficing for a potential refutation. A possible refutation by third parties does not need to be a blemish on the proof of academic competence. To this criterion belongs the possibility of retrieving the sources on which the design and its argumentation is based. The requirements of the design drawing as a document to be judged scientifically are further detailed in Chapter .


B.     The academic design should bring possibilities to light that are essentially new (‘invention’ or ‘find’). This novelty value should show by comparing it to an added, accompanying, inventory of similar existing designs in order to provide the person evaluating with the wherewithal for his task. The technical-scientific design should bring improbable possibilities, those not to be deducted by mere prognosis. With this, the novelty value exceeds new knowledge (discovery) of phenomena at empirical study, probable by themselves.


C.     It should be made acceptable that these possibilities are presently technically viable, at a future time economically and in any perspective as well as socially. The design should include a vision on the range of technical execution and social implementation in that perspective.


D.     The design should include an effect analysis (for an evaluation ex ante, see Chapter ) of this book). This analysis should minimally include a physical (spatial, ecological, technical) and a social (economical, cultural, political) effect in different perspectives. These effects may be intended in the first stage (potential of the site, intention, social need for the programme) and unintended afterwards. The effect analysis comprises particularly the unintended effects; for the intended ones, relevant during the comment on the design and the argumentation, would lead to circular reasoning. Unintended effects may be judged negatively afterwards in certain perspectives. They cannot be a basis for discrediting design and study competence. If demonstrated by the designer himself, on the contrary, it should be regarded as a proof of a scientific propensity. Additional illustration on effect analysis is given in Section .


E.      The intended social effect should be admissible in terms of ethics. Of the unintended effects the ethical admissibility should be checked.


3.8           Criteria for design related study ex ante

One should also distinguish – next to evaluations before and after execution – between evaluations of a design related study ex post and a proposal for such study to that effect before a design has been made. Judging a study proposal in advance is more difficult since there is less material at hand: it is just a promise of a study. In spite of that many study committees daily face the task of judging the potential and relevance of study proposals. Authorities distributing budgets always wrestle with the difficulty that a study proposal can not yet be judged on its result: the criteria of paragraph 3.7 can not be applied.


This book was written in the framework of the pilot project ‘The Architectural Intervention’; a number of workshops where teachers and students study and publish together.[22] The project proposals have been judged ex ante as a study proposal by the Methodology Committee of the pilot project for admission to the pilot project according to the following criteria.


A Affinity with designing

Affinity with designing can be shown from at least two images (photo’s, drawings), which are somehow comparable, or which in previous studies stood for reference or design model for an important field of interest from the participating researchers/graduate(s) and for the studio as a whole. The images may be a portrayal of different locations (at any scale), but also from the same location in two development phases. A correct way to do this is an entry in the Interactief Beeld Archief (IAAI, see the Internet site).[23]


B University latitude

University latitude can arise from a specification of the context and the perspective of the research, from participating disciplines and contacts with (inter) faculty research and graduating in the studio. An external referee can take on the rôle of an imaginary assignment initiator if (s)he is prepared to remain involved with the research/ graduation up until the final publication.


C Concept formation and transferability.

Concept formation with regard to design-orientated thinking follows the course of describing to accurate formulation in key-words. Concepts are defined, or, if they cannot yet be defined, receive a conditionalposition(see page 41). Implicit, not commonly accepted assumptions such as the supposedly self-evident conditional and causal connections are made explicit. Scale falsification and overlapping concepts are avoided. What is stated as desirable, possible or probable is differentiated as such. That which can be expressed in an image, is not solely verbally expressed. In this case an image is made accessible by means of an unambiguous legend or drawing code.


D Retrievability and Accumulating Capacity

A correct way to reach retrievability and accumulating capacity is a personal internet site for each individual researcher with respect to the research/graduation and a site for the researcher studios and projects as a whole.


Referring to other authors

Provisional literature lists should be equipped with a number of key-words per title, from which it can be deduced that the proposal makes an input, uses, assesses or attempts to reject.


Making one’s own publications retrievable

An accurate, distinct and sigificant title for the project and for the sub-researches within the project should be found. A determination of identity, not a solitary example but the placing within or beside an existing research, as well as a series of key-words, which reproduce the fascination, knowledge and the presentation of the researchers’ questions are required. Reference words or distinguishing features relative to the design in the drawings and sub-projects, allow them to be retrieved in an image archive by differently orientated researchers. A few of these key-words can be elaborated upon in an explanation, which reproduces the theme, the study question and the study approach.


E Methodical accountability and depth

Such accountability, of the way in which (using which method) one will arrive at a result, should make possible that someone else using the same method can (not: will) arrive at a similar outcome.

There should be evidence to which extent the study is aim-orientated or means-orientated, empirically orientated or orientated towards the development of means of design. In the first case the starting points must contain a problem proposition and an objective proposition. An hypothesis and a research method must be specified.

A collective starting point for means orientated study can be perceived as follows: a location or a category of locations should become more meaningful using the design - for whatever purpose - than the current interpretation. The hypothesis of means-orientated research is always ‘There are means of design for different purposes to come’, which must be proven by the design.

The depth of means-orientated research can be proven with at least two criteria of each drawing, whereby they can be compared. Their differences or transformation can be evaluated and an explanation of the manner in which (method used) they can be compared.

This explanation can concentrate on the framework, the research field to which the comparison is reduced (for example a building physics, history and functionality comparison). This can also be used in order to specify which internal or external study programmes will be linked to.


F Ability to be criticised and to criticise

Ability to be criticised (ability to be refuted) offers others the opportunity to selectively make progress by building upon technical scientifical know-how and knowledge (accumulation) obtained through study. Statements are only of a scientific interest when they are bold and do not solely use risk-free citations, self-evident aspects or even clichés, on the contrary, statements must question these. This daring must not only be apparent in the project design but in particular during execution.

Ability to be criticised can be shown from a readiness and initiative to expose the results in their consecutive phases, to publish them for instance on the internet in a refutable manner, and in this way open them to criticism in all phases of the research even though these phases are unripe. Drawings and arguments must not conceal their weaknesses.


G Convergence and limitations

·        A proposal for the nature of the end product is required with a summation of the sub-projects. Questions to be answered in the proposal are:

·          Who is ultimately responsible, who takes part, how often do they meet, what do they organise, how do they divide the common tasks, how is a synergy created, which facilities are desired?

·          How is it to be represented (on the website, in book form, in a conference)?

·          How do the sub-projects converge?


This methodology book tries to help.



[1] Participating in the DRS newsgroup is possible by mailing a message ‘subscribe drs’ to mailbase@mailbase.ac.uk

[2] The European Association for Architectural Education (EAAE) is an international organisation of European educational institutes in the field of architecture. Its aim is exchange of ideas and results related to architectural education and study. For the benefit of the EAAE conference on Doctorates in Design + Architecture a conference book (‘Book of Abstracts’) has been published. The proceedings comprise two volumes, edited by Theo van der Voordt and Herman van Wegen: Voordt, D.J.M. van der and H.B.R. van Wegen (1996) State of the Art, Proceedings of the Doctorates in Design and Architecture conference, vol. 1; Voordt, D.J.M. van der and H.B.R. van Wegen (1996) Results and Reflections, Proceedings of the Doctorates in Design and Architecture conference, vol. 2.

[3] Kuypers, G. (1984) ABC van een onderzoeksopzet; 2e dr.

[4]   Zeisel, J. (1985) Inquiry by design: tools for environment-behavior research.

[5] Jong, T.M. de and J. Achterberg (1996) 25 plannen voor de Randstad. This study compared twenty-five designs by re-design for one million inhabitants.

[6]   Vollers, K. (2001) Twist & Build, creating non-orthogonal architecture. See also Chapter 55 in the present book.

[7]   See Chapter 56 in the present book.

[8]   Bacon, Francis (1620) Instauratio magna; The Great Instauration. Bacon, Francis (1645) Novum organum scientiarum.

[9]   Kant, I. (1787) Critik der reinen Vernunft.

[10]            Kuhn, T.S. (1962) The structure of scientific revolutions. Dutch translation: Kuhn, T.S. (1972) De structuur van wetenschappelijke revoluties.

[11]            See: Kroes, P.A. (1996) Ideaalbeelden van wetenschap, een inleiding tot de wetenschapsfilosofie.

[12]            Readily accessible methodological books include: Baarda, D.B. and M.P.M. de Goede (2001) Basisboek methoden en technieken. Baarda, D.B., M.P.M. de Goede et al. (1996) Basisboek open interviewen. Baarda, D.B., M.P.M. de Goede et al. (2001) Basisboek kwalitatief onderzoek. A little less recent, but with more examples on the field of construction is the book by Korteweg, P.J., J. van Weesep et al. (1983) Ruimtelijk onderzoek: leidraad voor opzet, uitvoering en verwerking. Especially for constructors is Zeisel, J. (1985) Inquiry by design: tools for environment-behavior research. For a systematic approach to formal (plan)analysis see, for instance: Clark, R.H. and M. Pause (1985) Precedents in architecture.

[13]            Popper, K.R. (1963) Conjectures and refutations: the growth of scientific knowledge. Partly translated in Dutch: Popper, K.R. (1978) De groei van kennis.

[14] Lynch, K. (1960) The image of the city.

[15]            Werkgroep Beoordelingscriteria Ontwerpdisciplines (december, 1999) Advies van de Werkgroep Beoordelingscriteria Technische Universiteit Delft.

[16]            The concept modality has a well-described function in philosophy (Aristotle, Kant), logic (modal logic) and linguistics (verbs of modality) to express the difference between probability, likelihood, possibility or desirability. See also Chapter 23 on logic .

[17]            Jong, T.M. de (1992) Kleine methodologie voor ontwerpend onderzoek. This book further develops this conditionality in a technical sense. It is shown that conditionality implies also a sequence that was ascribed previously exclusively to causality. A provisional translation is available at: http://www.bk.tudelft.nl/users/dejongt/internet/008000000.htm.

[18]            Vitruvius and M. Morgan (1960) Vitruvius: The ten books on Architecture.

[19] In the case of a means-orientated study the design solutions are generated first, and next is studied which aims could benefit from them (e.g. ‘ designing as an art to seduce’).

[20]            BCO, Bestuurscommissie Ontwerpers- en korte Onderzoeksopleidingen (1994) Op weg naar promotie op proefontwerp.

[21]            ‘Culture’ is defined here as the set of tacit pre-suppositions while communicating; for example the meaning of the units of the legends in the drawing.

[22]            See publications of the Architectonische Interventie.

[23]            http://iaai.bk.tudelft.nl/