Overclock | AICA Survey Questions

Section 1 of 4

Technical Skills

How comfortably you use AI tools in your day-to-day work.

Question 1 of 10

Which best describes how you use AI today?

I chat with tools like ChatGPT or Claude for writing, research and analysis.I've connected my chat tools to external tools and data — Google Drive, email, Slack, MCP connectors — so AI works with them directly.I use AI to write scripts and code, and to build applications.I build agents and automations, and embed LLMs into applications through their APIs.I run a team of autonomous agents working continuously in the background and on demand as a system.

Question 2 of 10

How do you give AI the context and instructions it needs?

I type a request into a chat.I create detailed requests with extensive attachments, images, and reference examples as part of my requests.I routinely point my AI tools to external systems and sources to derive samples to help fill context, and routinely build up detailed context files over time.I write the system prompts, skills and tool definitions for the agents and LLM-powered apps I build.I design shared context for a team of agents — memory, retrieval, skills, permissions — so each one knows what it needs and nothing more.

Question 3 of 10

What have you built with AI?

I copy and paste the output from AI into various destinations.I have built reports, PowerPoint presentations, and spreadsheets.I have built applications with coding agents or AI-powered app builders.I've built applications with LLMs built in — calling model APIs for real features — or agents that do work on their own.I've built systems of agents that work together, deployed and relied on for real work.

Question 4 of 10

What's your experience with AI agents?

I haven't used agents.I've used built-in agents that act in my other tools for me, like deep research or agent mode.I've had a coding agent carry out multi-step work for me — writing, running and fixing code.I've built my own agents with instructions, tools and API calls.I've built teams of agents that coordinate as a system — roles, handoffs, shared context and review.

Question 5 of 10

How many AI workers do you have going at once?

One chat at a time.One assistant connected to my other tools, working across them.A coding agent working alongside my chats.Several agents I built, each on a part of a task, coordinated by me.I run a standing team of agents that hand off work, review each other and run as a system.

Question 6 of 10

Does AI keep working when you're not at the keyboard?

No.Only through scheduled features built into the products I use.A script I wrote with AI runs on a schedule.I run automations or cron jobs that call LLMs or agents to do work on their own.I run teams of agents continuously on long-running work, with monitoring and alerts.

Question 7 of 10

How do you reach your AI tools?

A chat window.Chat, plus AI built into my everyday tools.A terminal or coding agent as well as chat.Agents I built, which I reach from Slack, an app or an API.Several interfaces — terminal, Slack, phone, voice — that reach one coordinated team of agents and their shared context.

Question 8 of 10

How connected is your AI to your other systems and data?

I copy and paste between AI and my other tools.I use built-in connectors (Google Drive, email, Slack) in a chat tool.I've written scripts with AI that move data between tools through their APIs.My agents and LLM apps connect to systems through APIs or MCP servers.My team of agents shares integrations across many systems, with scoped permissions and audit trails.

Question 9 of 10

How do you check AI-generated work before it's used?

I read it over.I check outputs against my own files and data, pulled in through connected tools.I run and test AI-written code before using it.I build automated checks into my agents and automations.My agents review each other's work against set criteria, and people handle what's flagged and any high-risk action.

Question 10 of 10

How do you know whether your AI setups are working?

Gut feel.I spot-check outputs against the source data in my connected tools.I test AI-written code and scripts before relying on them.I keep test sets and rerun them whenever I change prompts or models in my agents and apps.I run evals across the whole agent system on every change — quality, cost, latency — and catch regressions before they ship.

Section 2 of 4

Experimentation Skills

How often you try new AI approaches and learn from what works.

Question 1 of 10

Beyond everyday chat use, how many hours a week do you spend building, testing or debugging AI workflows?

Question 2 of 10

Roughly how many hours have you spent building and experimenting with AI in total — not counting everyday chat use?

Question 3 of 10

What are the most ambitious things you've tried to get AI to do?

Answer questions, write text, make images.Work directly with my files, email or other tools through connectors.Write code or build a working application.Build agents that take real actions across systems — email, code, databases, APIs.Build multi-agent systems that run complex work end to end, often past what the tools officially support.

Question 4 of 10

How much of your real work have you tried handing to AI — done by AI, not just with its help?

Small pieces, like a first draft.Whole documents or analyses that I then finish myself.A few whole recurring tasks, now done by AI.Most of my recurring tasks, at least as a trial.Nearly every part of my work, keeping what held up — and now work that used to take other people's time, too.

Question 5 of 10

How much time have you spent getting AI and other systems to work together — APIs, MCP servers, integrations, data pipelines?

None.I've switched on built-in connectors.A few hours wiring up an automation tool or API.Dozens of hours connecting agents to tools and data.Hundreds of hours building and debugging integrations until they ran reliably.

Question 6 of 10

How much work goes into your prompt sequences — the chain of instructions, steps and context behind an AI workflow?

I type a request.I reword it when the result is off.I build multi-step prompt chains for important tasks.I version prompt sequences and test changes against each other.Hundreds of hours of prompt and context design, with measured improvement I can show.

Question 7 of 10

How often do you catch AI getting something wrong — a false fact, broken code, a bad action, a misread instruction?

Rarely; I don't use it enough to notice.Occasionally, when something looks off.Weekly, in workflows I've built.Daily across my agents, and I know their failure patterns.Many times a day across my systems, and I predict and design around most failure modes.

Question 8 of 10

How often do you hit an AI problem you haven't seen before — and solve it?

Never; I don't push AI that far.Rarely, and I usually give up.Monthly, and I solve some.Weekly, and I solve most.Several times a week, and I solve nearly all — often by building something new.

Question 9 of 10

What do you read, watch or follow to learn what's changing in AI?

General news, or nothing in particular.A few AI newsletters, podcasts or creators.Release notes and docs for the tools I use.Model docs, technical write-ups and practitioner communities, every week.Primary sources daily — lab releases, papers, source code, builder communities — and I share what I find.

Question 10 of 10

When you learn about a new AI capability, what do you do next?

Nothing, or make a mental note.Try it once in a chat.Try it on a real task within the week.Test it against my current setup within days.Test it on my own workloads within days, adopt it if it wins, and retire what it replaces.

Section 3 of 4

Architecture Skills

How you design, connect and govern agents, workflows and evals.

Question 1 of 8

What's the most complex human process you've re-engineered so agents do the work?

None; I use AI on tasks as they come up.I've used AI to speed up a few steps in a process.I've rebuilt one process of my own so an agent runs it.I've re-engineered a multi-step team process so agents own whole stages and people direct and review.I've re-engineered complex, judgment-heavy processes — sometimes stitching together slices of many people's jobs — into agents that run autonomously across thousands of runs.

Question 2 of 8

Much of a process lives in people's heads — how decisions get made and the unwritten rules behind them. How do you capture that for agents?

I haven't tried.I describe the task in a prompt.I write down the steps and rules for one process.I interview the people who do the work and turn their decision logic, exceptions and examples into specs and skills agents use.I've captured decision logic across complex processes into context agents rely on at scale, and I keep it current as the process changes.

Question 3 of 8

How do you evaluate an agent before trusting it with real work?

I try it and see.I test it on a few examples.I keep a set of real cases with expected outcomes.I run a test set with automated and model-graded scoring on every change.I've built an eval system — graded datasets, pass thresholds — that gates every release and has caught regressions before they shipped.

Question 4 of 8

Once an agent is running, how do you check its quality over time?

I don't have agents running.I find out when someone complains.I review outputs by hand now and then.I sample and score outputs on a schedule and fix the patterns I find.I've built a QA system that samples and scores runs automatically, routes failures to people, and feeds them back into the evals.

Question 5 of 8

When an agent run goes wrong, how do you find out why?

I couldn't tell.I rerun it and look at the result.I read the conversation or logs for that run.I log every step, tool call and input for each run.I trace every run end to end — prompts, tool calls, cost, latency, outputs — and search across runs to find and fix failure patterns.

Question 6 of 8

When you build an agent, how does it gather the context it needs to do the job?

I paste in whatever context I think it needs each time.I attach the relevant documents and examples up front.I connect it to the source systems so it pulls what it needs itself — files, databases, APIs.I design what it retrieves and when: search over curated sources, loading only the relevant slices at each step.I've built a context pipeline — indexed, permissioned sources with retrieval tuned and evaluated on real tasks — that agents rely on in production and that stays current as sources change.

Question 7 of 8

How do you keep people in the loop on a process your agents run?

I either check everything myself or don't check at all.I review the final output.I approve the riskiest step before it runs.My agents escalate low-confidence or high-risk cases to a person by set rules.I tune escalation from data so people see a small, targeted share of runs, and I track that share and the error rate as volume grows.

Question 8 of 8

How do you make an agent process reliable enough to run unattended?

I haven't.I fix things when they break.I think through what could fail before launching.I add retries, fallbacks and alerts at known failure points.My process runs against a success-rate target with retries, fallbacks, alerts and a kill switch — and has met that target in production.

Section 4 of 4

Strategy Skills

How you find where AI creates value and plan how it changes work.

Question 1 of 8

How well do you know your organization's top-line metrics?

I couldn't name them, or only the general direction.I can name the main metrics leadership tracks.I know their current values and how my area's work moves them.I've tied AI work in my area to specific movement in those metrics.I've redirected AI investment based on those metrics, and the results showed up in them.

Question 2 of 8

How do you find problems in your organization that are good candidates for AI automation?

I wait for ideas from colleagues or vendors.I notice tasks in my own work that AI could speed up.I assess my area's processes for volume, repetition, cost and risk.I've shortlisted candidates — including work that was never done because people were too costly — and delivered some.I've run structured opportunity discovery across the organization, can explain why each pick suits AI and why others don't, and the picks have delivered.

Question 3 of 8

Have you put together an AI roadmap?

No.I have a list of ideas.I have a prioritized plan for my own work.I've built a team roadmap with sequencing, owners and target metrics.I've built an organization-level AI roadmap tied to strategy — phases, owners, budget, metrics — that leadership adopted and that is being delivered.

Question 4 of 8

How well do you understand what AI costs to run?

I don't know.I know what our subscriptions cost.I understand token pricing and roughly what a task costs.I can estimate cost per run for a use case by model and volume.I model cost at scale — tokens, model tier, caching, retries, infrastructure — and have cut real production costs with it.

Question 5 of 8

Could you calculate the ROI of an AI initiative?

No.I'd describe the benefits in general terms.I can estimate the time it would save.I can build an ROI estimate: build and run costs, tokens included, against time, cost or revenue impact.I've built ROI models that drove go/no-go decisions, measured actual returns against them, and the initiatives paid off.

Question 6 of 8

How do you decide whether an AI project should go ahead?

I build it if it's possible.I go ahead if it seems useful.I go ahead if it supports a known goal.I go ahead only if a target metric and expected impact are agreed up front.Against explicit criteria I set — metric impact, ROI, risk — and I've stopped projects that didn't meet them.

Question 7 of 8

After AI work ships, how do you check whether it paid off?

I don't.I ask whether people like it.I look at whether people use it.I measure results against the target metric and adjust or stop the work.I track realized ROI across a portfolio of AI initiatives, report it to leadership, and move investment based on it.

Question 8 of 8

How do you make the case for AI investments?

I haven't had to.I describe the technology.I estimate the time or cost saved.I build a business case tied to strategy and metrics — and I've had one funded.I've secured major AI investment from leadership or the board with a projected return, and delivered against it.

0 of 36 answeredAbout 16 min