"Certain principles apply when drafting questions. There are three rules of thumb for determining which questions are acceptable: (a) ask only for information that you intend to use to make a hiring decision, (b) know how you will use the information to make that decision, and (c) recognize that the practice of seeking information that you do not use can be difficult to defend (Ford, 1993, p. 13).  These rules suggest that the best questions will emerge from a careful analysis of the position."

Tomorrow's Professor Msg.#1386 Interviewing Good Questions, Bad Questions

 

Folks:

The posting below gives some important advice on developing good interview questions for academic hiring.  It is from Chapter 12, Interviewing, in the book, Search Committees: A Comprehensive Guide to Successful Faculty, Staff, and Administrative Searches, by Christopher D. Lee, PhD, SPHR. Published by Stylus Publishing, LLC. 22883 Quicksilver Drive Sterling, Virginia 20166-2102. Copyright @2014 by Stylus Publishing, LLC. [https://styluspub.presswarehouse.com/Books/Features.aspx] All rights reserved. Reprinted with permission.

Regards,

Rick Reis

reis@stanford.edu

UP NEXT: What Do We Mean by "Writing"?

Tomorrow's Academia 

---------- 2,320 words ----------

Interviewing - Good Questions, Bad Questions 

 

Good Questions 

Good interview questions will illuminate a candidate's experience well enough to indicate his or her prospects for success in the position. Bad interview questions will provide no indication of these prospects and may even expose the institution to certain legal liabilities (see Exhibit #20: Interview Questions to Avoid). Generally, all questions should be related to the knowledge, skills, and abilities necessary to carry out the duties and responsibilities of the job successfully. Questions about professional competencies such as work ethic, decision making, problem solving, and interpersonal relations that indicate a person's professional character are also fair game. 

Certain principles apply when drafting questions. There are three rules of thumb for determining which questions are acceptable: (a) ask only for information that you intend to use to make a hiring decision, (b) know how you will use the information to make that decision, and (c) recognize that the practice of seeking information that you do not use can be difficult to defend (Ford, 1993, p. 13).  These rules suggest that the best questions will emerge from a careful analysis of the position.  

Asking the Right Questions 

The most important part of the interview process is asking questions. Therefore, no committee's work is done well unless its members formulate and ask questions that elicit responses that help to make distinctions among the candidates. The right questions help determine the right person for the job. 

There are several different types of interview questions, including informational, situational, behavioral, and case study.  Each question type can be asked in an open- or closed-ended fashion.  Open-ended questions allow a free-form response, while closed-ended questions limit responses to yes, no, or similarly brief responses. Open-ended questions are usually more effective in eliciting qualitative information from candidates that allows committees to better judge which candidate is best. 

Asking good questions is more difficult than it might seem. Many committees are tempted to dust off the questions used the last time a search was conducted and modify them as needed.  Yet, it is more prudent to ensure that the questions truly reflect what is expected of the position today, considering changes in technology, regulatory requirements, needs of the department, and other requirements such as those that support strategic initiatives.  All questions should be drafted with current needs in mind. 

There is an art and a science to asking questions, and committees must understand both when drafting questions.  Practicing good art involves asking questions the right way. Indeed, how a question is asked determines the type of information you receive and whether this information is valuable in making the decision about who will work best for your organization. Applying sound science is using the right type of question given the selection criteria you are evaluating. 

Here is an example of putting the art and science of asking questions to a test.  If you are trying to evaluate a candidate's ability to manage the financial affairs of an academic unit; you could ask the question any number of different ways: 

Q: Tell us about your budgeting experience.

Q: Tell us about your experience with managing the budget of an academic department. 

Q: What process would you use to establish a departmental budget for a newly formed academic division?

Q: Have you ever had to cut a budget in the middle of the year, and if so, how did you decide what to cut and what to keep?

Each of these questions will generate responses such as general factual information, detailed examples of financial prowess, problem-solving ability, or detailed knowledge about the mechanics of organizational financial management. 

A response to the first question might be too general to evaluate the degree to which a candidate has experience or so specific as to bore the committee with details. The second question might be more on target for evaluating a dean candidate's experience, yet it might not give you enough information without a follow-up question that asks for specific examples.  The third question is excellent if you are assessing a candidate's financial competency, and a full answer is likely only to come from candidates who have knowledge and understanding of and experience with the subject.  While the fourth question would be better used to evaluate a candidate's problem-solving and decision-making skills, and maybe even his or her approach to shared governance, it might fail to provide enough detailed information about the candidate's financial acumen. 

Exhibit #20 Interview Questions to Avoid

You cannot ask any question during an interview that relates to an applicant's race, color, religion, age, gender, national origin, disability, or genetic predisposition.  In some states, inquiries about a candidate's sexual orientation, married states, or veteran status are also illegal.  As a rule, you may ask only work-related questions.  The following questions are examples of those that should not be asked: 

Questions to Avoid 

* Are you a U.S. citizen?

* Where were you born?

* What is your birth date?

* How old are you?

* Do you have a disability?

* Are you married?

* What is your spouse's name?

* What is your maiden name?

* Do you have any children?

* Do you have child care arrangements?

* What is your race or ethnic origin?

* Which church do you attend?

* What is your religion?

The following questions should be asked only when there is a bona fide, job-specific reason to ask them.  If you ask them of one candidate, then ask them of all candidates for the same position. 

Acceptable Alternative Questions

* Do you have any responsibilities that conflict with the job's attendance or travel requirements?

* Are you able to work in the United Sates on an unrestricted basis?

* Are you able to perform the duties on the job description with or without reasonable accommodation?

* Have you ever been convicted of a felony?

* If hired, can you provide proof that you are at least 18 years of age?

* Do you have any conflicts that would prevent you from working the schedule discussed?

* What languages do you speak or write fluently?

* Have you worked under any other professional name or nickname? 

* Do you have any relatives currently working for this institution?

* Would you have any problem working overtime, if required?

* Would anything prohibit you from making a long-term commitment to the position and the institution?

Situational or Behavioral Questions 

Most interview questions are either situational or behavioral.  Though they differ in their purpose and intent, they can be equally effective if used properly.  Behavioral questions start with the premise that the best predictor of future performance is past performance.  Therefore, they ask the candidate to recall a past experience and share how he or she responded. Behavioral questions ascertain the nature of the candidate's past behavior, but they are only effective if each candidate has had similar experience and is not seeking a promotion.  An example of this question might be, "Give us a situation when a student vehemently disagreed with how you graded one of his or her written assignments and how you resolved that conflict." 

By contrast, a situational question attempts to determine how a candidate might respond in a given scenario.  This type of question can be used to evaluate experienced candidates as well as those who are changing to a position different from their current one, such as a faculty member becoming an administrator.  A typical example of a situational question is, "If you had a disagreement with colleagues over a faculty governance matter, how would you work with them to resolve the situation if you disagreed on the underlying facts of the matter at hand?"  Situational questions ascertain a candidate's response to a hypothetical or real-life situation that tests his or her ability to analyze and solve problems or make decisions. 

Information Questions 

Information questions are direct questions that gather specific information or verify facts or data.  These are sometimes called factual questions. Information questions ascertain the specifics of a person's education and career.  Often information questions can be answered with a yes or no or very simply, yet they provide essential information about the candidate.  Here are a few examples: 

Q: Are you willing to accept a non-tenure track position?

Q: To what position does your current job report?

Q: When you moved from Small Town College to Big State University, did you consider that to be a lateral move or a promotion, given that the duties and responsibilities were similar? 

Q: Why do you feel you are the best candidate for this job? 

The primary uses of information questions are to verify information provided in the resume, follow up on a previous question, or collect factual information.  These questions gather information; they do not necessarily test candidates on how they may think or might react in a professional setting.  Situational, behavioral, and case study questions are better designed to assess a candidate's knowledge, skills, and abilities. 

Case Study (Scenario) Questions 

The case study question assesses how a candidate processes information, makes decisions, and solves problems.  It also provides a glimpse into how a candidate might react to adversity, how well candidates "think on their feet," and how they speak extemporaneously.  Sometimes called scenario questions, case study questions present a challenge or dilemma that is not easily solved or may not even have a solution.  The committee then evaluates how the candidate processes information and might approach his or her work.  The following example illuminates this sort of approach: 

Q: What would you do in a dysfunctional department that was trying to balance a budget and could not agree on the priorities of the department, the curriculum of the department, or with the leadership's approach to solving this particular problem?  You have three colleagues who agree that A is the best course of action, three colleagues who believe B is the best course of action, and you as the department chair think that C is the best option.  Meanwhile, the university leaders think that none of the options proposed is viable.  How would you handle this situation?

Clearly there is no right answer.  Yet, how the candidate responds to the question gives you a lot of qualitative information about how he or she approaches work, works with others, thinks, and might work if he or she were employed by your institution.  Case study questions provide an unscripted glimpse into how candidates handle difficult situations and how they might respond to the unpredictable environment of work. 

Asking the Questions Right 

The key to asking the right questions is acknowledging that asking good questions requires forethought.  Appreciating the fine art of inquiry and approaching the process deliberately will improve the selection process.  Yet, a simple three-step process can virtually ensure success in selecting the best candidate.  First, you must have clarity about the concept you are investigating - that is, financial acumen or decision making. Second, you must determine which type of question is best for dealing with that subject.  Third, you must be careful to phrase the question to elicit the qualitative information you seek. 

Bad Questions 
In addition to advice that helps you to draft good questions, here are some tips for identifying and avoiding bad questions: 

* Long questions: Long and wordy questions are often a telltale sign that the question was ill conceived. 

* Generic questions: Generic questions are so general that they do not help evaluate the specific skills candidates need for a position.  Instead of asking the overused question, "What are your strengths   and weaknesses," ask, "If we were to call your references, in what areas of academic administration would they give you accolades, and in what areas would they give you critical feedback?" 

* Compound questions: Be careful of compound questions that have too many parts.  They can lack focus or clarity and be confusing. 

* Too many questions: Preparing too many questions and then asking the candidate to provide brief responses can undermine the committee's ability to ascertain the qualitative differences among candidates'  responses.  Holding a 75- to 90-minute interview instead of 60 minutes might help this situation. 

* Similar questions: Questions on related topics that are not complementary should be dissected or combined so that one does not answer or provide clues to the other. 

* Favorite questions:  Committee members who want to ask their pet question regardless of the particular position should be redirected to the position advertisement or position description for a more appropriate line of inquiry. 

The importance of good interview questions is too great to be handled haphazardly, so care and attention to drafting and delivering good questions is advised. 

The Final Question: A Summary

The questions you pose will determine the answers you receive.  Poorly worded questions can make a less-qualified candidate appear to be a viable choice and handicap those whose experience would normally set them apart from their peers.  The slate of questions asked of each candidate should be sound individually and should be complementary to provide a holistic evaluation of candidates.  Ideally, questions are based upon the description of the position, the job advertisement, and the committee's charge.  Yet, the best questions start with the notion that how you ask the questions matters. 

It is indispensable in the selection process to ask the right type of question to evaluate a candidate's background and experience, as well as his or her fit with your organization.  Just as there are many different ways of solving a problem, there are many different ways of asking questions - some better than others.  Expert advice for search committees is to use the right tool for the right job - situational, behavioral, information, or case study questions - as determined by your selection criteria.  Most often, you will use some combination of all four types.  Final question - "Do you have increased confidence in your ability to draft effective interview questions after having read this overview of question types?" Good luck on your search. 

----------------------------------------------------------------------------------------------------
TOMORROW'S PROFESSOR MAILING LIST
Is sponsored by the STANFORD CENTER FOR TEACHING AND LEARNING
----------------------------------------------------------------------------------------------------