Transcript: 'How AI Will Change Science Forever'

The transcript of AI & I with Alice Albrecht is below. Watch on X or YouTube, or listen on Spotify or Apple Podcasts.

Timestamps

Introduction: 00:00:59
Everything Alice learned about growing an AI startup: 00:04:50
Alice’s thesis about how AI can augment human intelligence: 00:09:08
Whether chat is the best way for humans to interface with AI: 00:12:47
Ideas to build an AI model that predicts OCD symptoms: 00:23:55
Why Alice thinks LLMs aren’t the right models to do predictive work: 00:37:12
How AI is broadening the horizons of science: 00:38:39
The new format in which science will be released: 00:40:14
Why AI makes N-of-1 studies more relevant: 00:45:39
The power of separating data from interpretations: 00:50:42

Transcript

Dan Shipper (00:00:59)

Alice, welcome to the show.

Alice Albrecht (00:01:01)

Thanks for having me.

Dan Shipper (00:01:02)

So for people who don't know, we've been friends for a long time. You are the founder and CEO of re:collect, which you recently sold to— What company bought you?

Alice Albrecht (00:01:14)

SmartNews.

Dan Shipper (00:01:15)

SmartNews and I'm excited to have you on here because you're just one of those people in AI that's actually been doing it for a really, really long time. And you've seen many different AI summers and winters. And I just find you to be incredibly smart and thoughtful and have built a business in the space that that you sold— You said it was an acquihire, so I just want to get into. what happened in the business you learned about running a company in this era of AI and kind of what's on your mind now. And I'll say for people listening I really liked re:collect as a product and we have not caught up since the acquisition. So, we're going to just do the conversation that we would normally do just hanging out together. So welcome to the show, give people a little bit of an introduction to the product and the acquisition process. And then we'll take it from there.

Alice Albrecht (00:02:10)

Yeah well, I'm excited to be doing this. We have not gotten to catch up, so this is good. So yeah, re:collect was—which is interesting to talk about it in the past tense now, but it feels, bittersweet. I'm proud of what we did and I'm excited about it, but yeah, it's hard just because I sat with it for so long, which I'll go through sort of the journey of building in this time of AI and how everything feels really new. I don't know if time feels like it's accelerating faster than it would have if I built this company at another time. If it was a software company five years ago with no AI, I feel like it would have been a different experience or a different journey. In any case, the re:collect was really focused on knowledge workers. You got to try out different versions of it over the years, but our main goal was, can we take all the stuff that you're consuming, could we connect it for you in this really personal way that your mind would? And then how do we make use of that for knowledge workers? So, we started out with this tool and we had a few different interfaces over the years we worked on it, but you could recall what you were thinking if you were doing writing or research. This is sort of pre-ChatGPT times. And then earlier this year, wow, yeah, this calendar year we had shifted into, don't tell us what you're thinking about—tell us what you want to accomplish. And then, we'll bring all the materials to you. We'll do the synthesis for you as needed, or we'll sort of alter those materials in a way that's useful to you as a knowledge worker. And everything we were doing was really aimed at how do we make all this stuff that we have access to accessible and useful for you? With the goal of enabling human intelligence really rather than the machine intelligence piece of it.

Dan Shipper (00:03:56)

I just resonate so much with that mission or that goal. That’s one of the things that got me so psyched because I'm a little bit more of a latecomer to AI than you. And so one of the things that got me psyched about it is, I feel like I've been such a nerd for tools for thought or technologies that expand the way we think or the way we see ourselves or the way that we create things and what we understand ourselves, all that kind of stuff. And it just feels like there’s so much there and you were playing in that space and, I'm kind of curious, what did you learn about trying to build a business there that you didn't know before?

Alice Albrecht (00:04:50)

Yeah, I think— A couple of different things. When I started— I left my full-time gig to work on this in 2019. So things were really different at that point. BERT and ELMo, these early models that came out, it’s all opportunity in the ideation phase. Okay, we have a capability, what do we build with it? What's useful in the world of this capability? What's possible now that wasn't. That was an interesting process. It took, I'd say, a year and change probably with Covid to be stuck in there and all of the chaos that surrounded that. But the journey of building a business for it, I think was, it had these separate phases. One of them was, I would say, we have this really nascent technology. Nobody really understands it. I'm trying to explain what's possible to do with it, but the models really weren't quite—this is GPT-2 maybe—they weren't amazing yet. I could see potential, but that was hard. I think once the models got a bit better and especially after ChatGPT launched, everybody woke up and was like, wow, this is really cool.

And then we had this middle phase as a company where I still was a bit skeptical about, okay, do we shove this into a product and have it generate things? Do we trust the models to do that in a safe way in the same way? And I thought, no, and so we had this middle phase of company building where we were launching a consumer product. We wanted it to be something that was useful for people, but also navigating all of the, I guess, oh, we could do that, but is that really in line with what we want to do? Or is that just because something new came out with a model and we could veer in that direction? That was interesting to navigate. I think, on the whole, now having gone through the whole cycle, if I had to do it over again, or had the opportunity to do it over again, I would have launched the product much earlier. I would have let the messiness be okay with customers and just see what happens with that. And I think we did a bit of trying to protect that and say, okay, we want to have this integrity. We want it to feel like it's a really seamless experience. We don't want them to really feel the weirdness of using these AI tools. And then the third phase is really as these tools are way more widely adopted, they're in everything. There's this huge explosion in this space trying to think about, okay, what is the actual application other than retrieval? Because retrieval was such an easy one. And we were doing a lot of retrieval stuff in the beginning, but then what happens after that? Once you retrieve a bunch of things, what is useful to do with those things? Yeah, so it was an interesting tri-part journey, I'd say.

Dan Shipper (00:07:38)

That's interesting. I definitely think that's such a common thing and it's so hard. It's easy to say and hard to actually internalize. It's just, ship something faster that you're not as proud of and just seeing what happens, can be very scary because obviously you want to make the best product possible. And it's a weird thing. Sometimes making the best product possible is starting with doing the messy thing or whatever. It's just weird. It's a weird thing to get into your brain. I think one of the things I'm interested in is, when you started it was like BERT and maybe GPT-2-ish days, what made you be, okay, this is happening now?

Alice Albrecht (00:08:33)

I think I willed it to happen, honestly, even going out, we did the typical VC fundraising process, too, and so, there was no moment where I was like, okay we finally hit it. I was like, no, no, this is going to happen. Come hell or high water. Even if I have to make this true—so yeah, I feel like it was really convincing other people that no, no, no, these embeddings I'm talking about are really, really important. And that was hard.

Dan Shipper (00:09:03)

Okay. And what do you think about— You wrote an article in Every a while ago and I read it and edited it a really long time ago, so I only have some of it in my brain, but basically I just feel like you have a really strong and clear thesis for how humans and AI can work together and what that should look like or could look like. And it's, it's a sort of cyborg hypothesis and I wonder if you could lay that out because I think it's a big question on people's minds. Where are we going if we're going to use this as tools? How does it work? And also keep us being humans and employed. And then I'd love to talk about how that has shifted for you over the years building this company.

Alice Albrecht (00:09:55)

Yeah, and it certainly has shifted, which has been interesting to kind of catch myself in between you saying these statements in the world and then something changes and you're like, oh, you'll have to readdress or reassess the whole situation.

So the cyber piece I'm still pretty hell-bent on. I think this has been my schtick for a long time now, coming from human studies to working with machines. I have this belief that humans are really kind of amazing creatures and that, if we're going to use the technology, it should be to augment them rather than to just build an AGI. The goal for me is not some sort of super intelligence that works without us. How do we start to connect this technology to humans? I'm really finding those touch points. I think knowledge is one of those. So if we think about knowledge work in general, a lot of it is consuming, distilling information, understanding and really kind of propagating that information out to other people, other knowledge workers in those systems.

And so, for me, I think a lot about how we can use AI or machine learning or these other technologies to make that process easier for humans so they can do the creative piece. And I think that's where we get to how do we all stay employed? I'm a big proponent of lowering the bar, raising the ceiling, I guess. So, what's possible we don't even truly understand quite yet. But if we can find the touch points where it makes a lot it makes that human work a lot easier. And I don't think we're quite there. I don't think chat is the right interface for this. I don't think the models have the right context for it. I say this again and again, I still think we're going to need some sort of biometric feedback piece in there to really make this work. And in that case, it's about finding where humans are not deficient, but where they kind of struggle from their evolutionary constraints, where technology can come together. And it's not creating artificial eyes. That's pretty cool, but that's not exactly my thing.

Dan Shipper (00:12:05)

Yeah, that makes sense. I was laughing earlier because I feel like we have this recurring sitcom or soap opera or something that every time New York Tech Week comes around, we're on a panel together. And the first year it was me and you and it was right after ChatGPT launched it and it was a couple of other people. And, yeah. I won't say who or whatever, but there was one person on the panel where me and you were just looking at each other the whole time. You're like, what is he—? What did he say?

Alice Albrecht (00:12:32)

What is happening here?

Dan Shipper (00:12:35)

Yeah, that was really fun. And then the next year we were on another panel about AI and creativity or something like that. And we were on different sides of the debate because we're talking about chat interfaces and whether or not chat interfaces are the future of interactions. And like you said just now, you think the answer is no. I was defending chat and I'm curious how that position has evolved for you and, if you could refresh my memory, why do you think chat is not the right interface?

Alice Albrecht (00:13:10)

Yeah, I’m trying to remember the argument I made on this panel. I probably won't be able to recall it exactly and I would also love to hear how your position may have changed over time. But, I think, language is a super powerful tool. We're using it right now to communicate with each other. We consume it in this way. I don't think that it is the most natural way to access things like this. I think we've actually moved farther in the search and probably since that panel too, with all of this agent stuff happening, where we have this kind of thing that does work in the background for us, but it intuits what work to do a little bit more. And it accomplishes that without us saying hello, robot, please, blah, blah, blah, blah, blah and I think the only way I've come maybe a little closer to the chat piece is I've used it more. I find myself in new spaces where I'm like, okay, I just started a new company, I'm in a new area where I'm thinking through things. It has become a little bit more useful for me to say, okay, I'm thinking about this, what are the pros and cons? And right now I have colleagues that are in Japan. We're 13–14 hours removed and so it's helpful for me in that sense. It's still not my main interface, though, so I think I'm doing more coding too. And so I think the main use for me right now is code. And that's sometimes chat.

Dan Shipper (00:14:45)

Are you using Copilot, Windsurf, or Cursor? Or like what are you doing?

Alice Albrecht (00:14:50)

So I'm coming back to Cursor. I had used it really early on. It was really buggy. And I was like, this is ruining my get-up. I can't use this. This is terrible. I'm coming back to that. So I use VS Code. I have Copilot for all the regular things. I do some chat. I really like the Claude Artifacts. That has been a really big game changer for me. And this is where maybe my chat argument does hold up. So it generates these—basically—little maps of things. You can have it do these mermaid diagrams as one of the artifacts. And I think really visually I draw things out a lot. I'm famous for having this remarkable thing with me everywhere I go and sharing these—I think I put these in one of my articles I published. This is how I think. I don't really think in this chat back and forth situation, so generating that has been interesting for me, too, as a way to collaborate in a non-text way.

Dan Shipper (00:15:47)

That is interesting. So for me, when I talk about chat, I would include voice, video and the sort of back and forth. But maybe it's more properly limited to just text and I think probably the reason I like chat as interfaces is I'm just so verbal. So that just makes a lot of sense. But if I try to turn my own personality into a general truth about what's good, which I think is typically how things work if I try to do that consciously, I think the reason that I that I like chat as an interface is it allows you to push forward along many different dimensions simultaneously where a lot of other software interfaces are either on or off. Or it's along one axis at a time or something like that, which is quite useful. Sometimes for refining something or for processes that you're doing where the dimensions that you're improving along are really well known and really well understood and it's sort of repetitive but then there are a lot of other processes, particularly creative processes, where you're trying to explore space along a bunch of different dimensions and you don't really know what the dimensions are beforehand. And I find that to be quite good for that exploration process and for the refinement process of, okay, you’re producing this thing, now I want you to push it in this way or define what that was before. But now I see it. Now I know.

So a really good example is we've been incubating products inside of Every, which is really cool. I would love to tell you about it. And one of the products that we incubated is this product called Spiral and it helps to automate a lot of the repetitive creative tasks that you do if you're running a company or you're a marketer or you're a creator. So an example would be, for this podcast, I have to take this podcast and turn it into a tweet to get the episode out. And that's a very repetitive process because I kind of have a format that I know works. I have a good first line and then I have a couple of bullet points about things like, what are the key topics that we discussed that I think are interesting or whatever. And I realized that Claude is really good at doing this. If you give it a few short prompts and you give it a bunch of examples of podcast transcripts tha turned into tweets, then it can do that over and over again. And it gets you like 80 percent of the way there. So we built Spiral, where it's basically a few-shot prompt builder. So you can make a Spiral for turning podcast transcripts into tweets, or you can make one for turning blog posts into LinkedIn posts or turning whatever release notes into a product announcement or whatever and I'm giving you this very, very long-winded explanation because one of the things that we found is in the in the Spiral interface, when you have a Spiral—let's say we take my podcast transcripts to tweet example—you just paste your transcript and you press run and then it gives you a bunch of example tweets that you can try. And one of our biggest pieces of feedback from people wanting an improvement is, I just want to chat so that I can say, this example was good—do more of that. You didn't really think that that would be the case. We had ideas for, maybe we could do sliders or really what you should do is go back into the Spiral creation flow and modify the prompt a little bit and make the prompt a little better or something like that.

And what we're finding is the natural thing to do is sort of you run a company, it's sort of when someone that's reporting to you comes to you and it's like, I did the thing that you asked for and you're like, okay, this is great, but here's a couple of things I need you to do. That's a very natural way to kind of push something in a sort of multidimensional, somewhat unknown space. Does that make sense?

Alice Albrecht (00:20:25)

It totally makes sense. And I think that using AI today is really good for this. Here's the first draft. Be it creating the tweet for me, writing some code for me, whatever, any kind of production thing. In that process though, I think there's this very interesting teacher-student relationship, which shows up. I think there's some new stuff around this, around training tiny models, based on big models, and those are really cool.

But I think it's interesting—there's two interesting pieces there. One is the human in the loop. And this, I think we do usually agree on, which is the human creative piece of this where you have the judgment and saying, nah, this one's not quite right, but here, change this one. Or, I can choose from this list. Awesome. The Claude model or whatever using it doesn't have enough information or intuition or something that seems a little bit fuzzier in there to choose. Okay. This is the best one. Just roll with it. I think that's going to continue to be the case.

And then the other piece in there, though, is this knowledge sharing. So in a sense, when Claude or whatever model you're using outputs, these sweeps for you get a little bit of a peek into how it's, quote unquote, thinking. It generated these things. You can see these, it got it kind of right. These are kind of wrong. You could give it feedback, but you're getting insight into that. And when you give it data, it's not learning in real time with the few-shot pieces. But maybe at some point it could and I think the more that you get a system that knows a little more about what and how you— I don't even want to call it preferences because it's so squishy. You can't learn these preferences easily but the more that you and the model can get to a shared understanding of what the other thing knows and can fill in the blanks. You can say, oh yeah, you wrote that one, but you forgot, or you didn't know, or, by the way, actually add this piece because this is critical in there. So I think this way of interacting and that even to me, I think the original like chat interface of ChatGPT where people are like I can talk to the thing and everyone likes chatbots. This is already beyond that when it creates artifacts. It's not a conversation. It's not like, hey, I think you should generally do. It's like, no, no, here's your thing. Here is your output and that's gotten really interesting.

Dan Shipper (00:22:49)

Yeah, that is, that is interesting. I was arguing against chat interfaces and then the product that I built was like a chatless interface, which is, I think it's your point, that reducing that down is actually— It can be helpful. And then we need to bring it back in some form, but there's interesting trade-offs to that. And the form that we're bringing it back is definitely not a totally general interface. And that's the only way that it can compete with Claude. You can do the same thing in Claude. It just has to be very specific to your purposes. I've been thinking about doing something. And something about the shape of it makes me feel like you would be into it or have interesting thoughts on it and whether it would work and what I should think about. So, I'd love to talk about it. You said something earlier that reminded me of it, but I forgot what it was. So let me just lay it out for you. I'm pretty curious.

Okay, I don't know whether we've talked about this before but I have OCD. One of the things I've been thinking about or lightly trying is I wanted to see if it was possible to wear a WHOOP, so to take my WHOOP data and be able to label from the graphs whether or not I was experiencing OCD on a particular day.

And so far, my answer to that is kind of, maybe. But, it actually probably needs more context to know. Because what a spike means on a stress draft and a WHOOP and a WHOOP can mean a lot of different things depending on what's going on in the background. And what's really cool about these models is now they know enough to be able to take the context and use that. And so, I'm not far enough along yet like another thing I'm going to try is I'm starting to do daily, two-minute video journals and there's this emotion-labeling AI called Hume and I'm going to see that it can label it. So, I don't know tha, there's some interesting things there that I think about. I feel pretty confident there's some combination of data where I can get from data to label to be, yes, he's having OCD symptoms today or no, he's not. And when I'm there, the big question to me is, will it be possible to predict, let's say a day in advance, whether or not I will start to experience OCD or whether or not I'm in a sort of OCD phase, whether it'll go away because I think once you can predict it opens up lots of interesting things and so my thought for how to do this is to basically do a bounty. And do $10,000 if you can predict my OCD, here's a data set and let anybody—because everyone you can have—you can use o1-Pro for $200. Anyone can do this basically now in a way that they couldn't before. And a) see if that works and then b), maybe build that into a sort of Kaggle but we're now anyone's a data scientist. And there's probably a lot of things that you would be into about this, but or have thoughts on but the thing that I'm kind of interested in— I think making predictions about and maybe we talked about this a bit, but I think making predictions about whether or not I'll have OCD is a form of science, but it's a form of science that like a scientist would never do because it's an end-of-one experiment and you're not actually looking for a causal explanation. You're just predicting so it's completely taboo to the establishment of research, but it's completely incredibly useful. And I think now doable where I think we should be doing a lot more of these end of one things and pursuing more predictions over underlying scientific cost explanations. And I just thought that that would get your brain going and you'd have interesting things to say.

Alice Albrecht (00:27:13)

Totally. And I appreciate you reigning it in too, because I have all of these pieces I can talk through. Yeah, I think the end of one thing is interesting because, if you came to me and you're like, I am in great pain from this. You were like, I am suffering. And as my friend, I would be like, gosh, how do I help you? And if I thought I could slap together a model, you got your WHOOP data—great. It'll take me whatever amount of time, but I could just do this. I wouldn't actually go to one of the large language models straight away. My brain would say, we need a predictive model. We need to understand the data sources. So I think it would be an interesting combination of looking at the actual research. So there is research out there on OCD—I'm not an OCD researcher.