Transcript of the talk, from a Whisper transcription of the audio. Auto-generated: speaker names in particular are unreliable.
========================================================================

# Now Hiring: Firm-Level Adoption of Generative AI in China

Authors: Lixing Li, Zhencen Liu, Anqi Wang, He Wang, Shu Yu  
Discussant: Kristina McElheran  
Video: https://www.youtube.com/watch?v=pwODfPHl7WY&t=4222s


## Talk (01:10:22 – 01:30:38)

[01:10:28] now hiring thank you the authors I would like to present a paper non-hiring firm level adoption  
[01:10:36] of generative AI in China um so this is actually very much work in progress we just rushed to submit  
[01:10:42] the paper to the workshop and then luckily we got in um so this is a background paper for the  
[01:10:48] word bank uh so sorry the word development report 2026 on AI the findings interpretations  
[01:10:54] and conclusions expressing this paper are entirely of the authors they do not necessarily  
[01:10:59] representing the views of the word bank uh or is affiliated organization so in the following  
[01:11:12] 20 minutes I would like to go through the following items so so starting we would motivate us  
[01:11:18] to do this what do we want to do in this paper is that we want to understand the firm level  
[01:11:23] adoption of generative AI among Chinese firms so we um so this is actually um the motivation is very  
[01:11:32] simple so China has emerged as a global force in terms of AI advancement however we know quite  
[01:11:38] little about how the the speed and depth of AI adoption among its firms so far the empirical  
[01:11:45] studies have been predominantly focused on high-income countries such as the US or Europe  
[01:11:51] where the economic and market structures are quite different we think those conditions could shape the  
[01:11:56] capacity incentives for generative AI adoption therefore um you know it is very important to  
[01:12:03] understand how AI diffuses or cross firms in a developing country like China um yeah I don't  
[01:12:10] know why there is a black bar over there but what we want to say is that um this is the  
[01:12:15] contribution of our paper we try to be the first comprehensive job posting based picture of generative  
[01:12:21] AI adoption among Chinese firms covering both listed firms and non-listed firms so listed firms  
[01:12:27] are very special because they are sizable and they are technically very advanced so if we are  
[01:12:33] only focusing on advanced uh those kind of listed firms we cannot draw a very um kind of  
[01:12:39] representative picture of the business landscape in China therefore we also extend our analysis to  
[01:12:45] cover non-listed firms so that we also know the small firms how do they adopt in the context of  
[01:12:51] China um so here is a quick review of all the existing measures on AI adoption um we so um yeah  
[01:13:01] so so the the most um you know common ones that you can first think about will be the  
[01:13:07] survey based approach where you go to ask the workers or owner of a company or whether  
[01:13:12] you are using AI and where you are using AI the problem with this literature is that these measures  
[01:13:18] tend to vary between studies um they are sensitive to the scope of technologies that is covered in a  
[01:13:24] study the wording of the questionnaire the sampling and awaiting method that are used and then  
[01:13:29] whether who is the respondent if you are asking the worker and then the worker will say no  
[01:13:33] I'm not using AI to do my job of course everything is verified by myself I'm not using it um  
[01:13:39] they're causing some kind of under reporting from the worker side but when you are asking the owner  
[01:13:45] for some strategic purposes they tend to overstate the importance of AI for their production  
[01:13:52] we also look at literature I think some some of the studies in today's workshop we'll also  
[01:13:58] talk about the patent based approach the patent is a based approach has this limitation  
[01:14:03] that it tends to identify AI developers instead of ordinary AI adopters and we would like to also  
[01:14:10] know what is the adoption patterns among those ordinary adopters um another measure would be  
[01:14:17] using the self-disclosure approach where you do text analysis on firm website attacks from  
[01:14:24] firms websites or annual filings the problem with that is also the potential cheap talks  
[01:14:31] where the companies overstate the AI importance in their business so that signaling their  
[01:14:39] technical advancements more recent studies have been using conversations captured by chatbots to  
[01:14:45] understand how AI has been used for various tasks the problem is that it's very platform  
[01:14:51] specific and it didn't really represent the firm's decision on how they adopt AI the last  
[01:14:58] so we highlight here which is the approach we use which is the job posting approach  
[01:15:02] there are a few advantages of this approach first we know that it is a costly real investment it  
[01:15:09] captures a firm's decision to invest in AI related human capital or organization capital  
[01:15:16] it has a very high frequency so if you are looking at a technology shock that you can see  
[01:15:20] the responses in the following weeks instead of running surveys after six months or something  
[01:15:25] and it's also applicable to all type of firms we can also know which function and what type of roles  
[01:15:31] that has adopted AI of course it has limitations so it will miss off the shelf adoption or  
[01:15:38] rescaling part of the existing workforce and there's a little bit of a gap between intended  
[01:15:44] adoption and realized adoption intended hiring versus realized hiring but for our purpose  
[01:15:51] we take this approach and we define a firm as a generative AI adopter once it first posts a  
[01:15:57] vacancy explicitly requiring generative AI skills we also computed posting intensity to  
[01:16:04] understand the extensive margin of the adoption we get 72 million unique job postings from three  
[01:16:12] major chinese job boards over the period 2015 to 2025 we match them to 5 000 listed firms  
[01:16:20] 2.1 million non-listed firms we also developed a kind of a keyword list a higher typed keyword list  
[01:16:29] and then match those keywords into three categories and we have the generative AI two skills  
[01:16:34] generative AI technical terms and early pre-charge btai we think it is very important to make the  
[01:16:40] distinguishing between the two skills versus the technical skills because the two skills  
[01:16:45] basically is applicable to all business functions and different type of roles within an organization  
[01:16:51] where the technical terms have a little bit of higher technical kind of entry barriers to workers  
[01:16:58] and then it can be more applicable to certain roles and technical functions instead of applicable  
[01:17:03] to all and we also try to use a pre-charge btai terms to understand whether the firm  
[01:17:12] to serve two purposes one is to make a benchmark to understand whether the diffusion speed of  
[01:17:17] generative ai is similar towards the early AI adoption the another thing is to use that to  
[01:17:25] capture existing AI talent within a company to understand whether the existing capital  
[01:17:30] could speed up the adoption for a new technology like generative AI we also try to do some  
[01:17:36] methodological improvements here so one thing is that we are pulling drop drop postings from  
[01:17:42] different drop boards so we did our own deduplication process we tailored the keywords to the china  
[01:17:47] context because we also included a lot of domestic apps and also use keywords in both  
[01:17:54] chinese and english we also were developing some algorithm to deal with the false positives  
[01:18:00] so for instance llm large language models as we know it represents generative ai it's a generative  
[01:18:06] ai term but in the postings law firms also use llm to specify a master of law degrees there are  
[01:18:14] also this g an which is also a kind of a generative ai technical term but in the context  
[01:18:21] of china it could be main pinyin which could be very random so we are developing also we are  
[01:18:27] taking this positive thing quite quite seriously we also try to understand adoption by business  
[01:18:37] functions by using llm to classify them into technical functions versus non-technical functions  
[01:18:44] to understand you know how those diffusions is happening within a company  
[01:18:51] so what do we find these findings are kind of basically stylized facts we started we  
[01:18:56] find that first of all generative ai adoption picked up steadily after release of chat dbt it  
[01:19:03] actually accelerated sharply right after deep seek become available in january 2025 the adoption  
[01:19:11] is fast but quite uneven so at the end of 2025 so 21 percent of the listed firms have adopted  
[01:19:18] a generative ai only 0.2 percent of the non-listed firm have listed a posting requiring  
[01:19:24] generative ai related skills we also did some regressions to see that you know what are the firm  
[01:19:30] characteristics that are positively associated with generative ai adoption we find larger firms  
[01:19:35] firms in market service sectors and if they have earlier ai talent tend to adopt a generative ai  
[01:19:42] faster and more we also look at adoption sophistication so if you are adopting technical  
[01:19:49] terms then we are thinking you are it's more sophisticated adoption if you are using it for  
[01:19:54] more technical business functions then it's also more sophisticated adoption we find adoption  
[01:20:01] sophistication is higher among listed firms and also in market service we also did a little bit  
[01:20:08] of a plumber analysis to look at the job posting that has requirements for generative ai related  
[01:20:14] skills and look where they are we find very interesting the top ones is not only ict specialist  
[01:20:20] it also covers some non-ict roles now okay so what kind of data are we using so we get data from  
[01:20:29] three major online drop boards in china so if you don't know it online drop boards is about 70  
[01:20:37] percent of the recruitment activities in china are done on those online drop boards and these three  
[01:20:44] major ones cover a majority of the online job postings so we have full access to job postings  
[01:20:52] from wuyao job 551 jobs yeah we in total we have 67 million unique postings and 63 million  
[01:21:02] of them were able to match to a business registration and then for the listed firms we  
[01:21:08] pours job postings from three boards in total we have about 17 million role postings and  
[01:21:15] after due duplication we have about 15 million of them in total we are dealing with 72 million  
[01:21:21] unique postings the postings as you may know that they have a lot of rich information about  
[01:21:27] you know the firm's name posting time location job title job requirements and responsibilities  
[01:21:35] so this is an example i think people who are familiar with this knows how to although this is  
[01:21:40] in chinese chinese so if you look at on the right it is the firm name the which firm is making  
[01:21:46] this job posting and on the top is job posting time the title of the role that they are recruiting  
[01:21:52] and then a lot of structural texts or unstructured texts about job requirements and here it says  
[01:21:58] one thing says familiar ways using ai facility writing using tools such as chat gbt so that's  
[01:22:06] exactly what we are trying to do we try to look for those kind of texts and identify the ai  
[01:22:11] generative ai related keywords and understanding whether this posting is requiring generative ai  
[01:22:17] skill or not to give you an idea about the sexual distribution of the firms so the blue bars are  
[01:22:26] showing the listed firms you can see it is quite concentrated manufacturing 70 percent of the listed  
[01:22:31] firms are basically manufacturing firms the the sexual distribution for non-listed  
[01:22:36] firm are much more even they they have a stronger presence about the service sectors  
[01:22:42] so wholesale retail these kind of three top sectors can account for 50 percent of the non-listed firms  
[01:22:48] in our sample it is very important to know the distribution sexual distribution of those firms  
[01:22:54] to interpret the results and here we classify the the keywords into three categories the first  
[01:23:03] one is the generative ai tools where we mention common tools such as chat gbt deep-sea q1  
[01:23:11] they are very specific they are used to capture entry-level cross-functional skills  
[01:23:17] generative ai related skills the second category is more specialized technical terms such as  
[01:23:22] transformers fine toning rag prompt engineer they are used to capture advanced generative ai  
[01:23:28] skills and which tend to be concentrated in more technical functions we also use pre-chat  
[01:23:35] gbt terms such as deep learning computer vision machine learning to capture early ai skills  
[01:23:42] as i mentioned the listed have been tailored to the china context and we find about half a million  
[01:23:48] postings with related keywords and here we're still working on how to reduce the false positives  
[01:23:55] and then i hope in the next round of this paper we can show the algorithm that we develop over  
[01:24:00] there we show a dummy that is a very generative ai skills over total number of postings we also try  
[01:24:17] to assign each posting to one of the five functions technical production sales administrative  
[01:24:24] and financials here i want to highlight is that in in when we are designing the pipeline we also  
[01:24:30] take the context of china into consideration so for instance for the semantic embedding we  
[01:24:35] are using a transformer model that was pre-trained in in in the context of chinese language and a  
[01:24:41] keyword policy was also we also tried a different type of ai models to see the to do the lm reasoning  
[01:24:49] and we find the keyword policy was doing a quite good job in interpreting the job posting  
[01:24:56] here is two charts to show you the emerging keyword trends and so the left one shows you  
[01:25:03] the two keyword word type gpt is due to the most popular keywords among all the tools however  
[01:25:16] lot of the domestic tools are catching up in 2024 2025 deep seek appeared in 2025 is already the  
[01:25:24] second most popular tools among the list for the technical terms which is a little bit hard  
[01:25:30] to read here but the interesting thing is that in the very beginning so the ai gc which is the  
[01:25:35] ai generated content was one of the most popular technical terms on the list but it was overtaken  
[01:25:42] by rag and other kind of terms were suggesting that the the usage of this technology has shifted to  
[01:25:50] simple content generating to a more embedded productive use of it and then we do we show the  
[01:25:58] intensity of the posting requiring the skills we you can see it is rising very fast the blue bars  
[01:26:04] that's very interesting is not only the gap is widening between the listed and non-listed firms  
[01:26:15] we can also see the technical increase is driving the gap between the two a little bit more intensively  
[01:26:22] i'm really yeah slow in terms of talking about this okay so i actually have lots of slides  
[01:26:29] so i'm going to be very fast here so this is showing you the cumulative adoption rate  
[01:26:36] over time what we i would really like to highlight is that the the the two gray dashed line here so the  
[01:26:43] first one is chatted chaff gbt and then the second one is the deep seek moment you can see the the  
[01:26:49] adoption rate has been increasing over time but what is very interesting is that around the 2025  
[01:26:54] you you can there's a notable spike in terms of the pick in terms of the adoption so by the  
[01:27:01] end of 2024 about 11 percent of the listed firm adopted generative ai tools sorry generative ai and  
[01:27:07] it jumped all the way to 15 in around january 2025 or february 2025 and we noticed that adoption has to  
[01:27:16] be you know higher for for firms in market service sectors here is another factor i would  
[01:27:25] like to draw your attention to is that we are in line with the existing literature we find larger firms  
[01:27:31] tend to adopt um are more likely to be adopters and it happens both in the listed firm and non-listed  
[01:27:37] firms and here we differentiate the listed firm by their ownership we find overall the rate is  
[01:27:44] quite different the same for so e's and non-so e's but this is mainly driven by a size effect  
[01:27:49] that so e's are much larger than non-so e's we also see the intensity has been increasing what is  
[01:27:56] interesting here is that the intensity has been much higher on the non-listed firm side than the  
[01:28:00] non uh than the listed firm because listed firms are much bigger so our interpretation that they  
[01:28:05] have capacity to absorb a little bit more uh in uh through hiring while the non-listed firms is  
[01:28:11] really a costly investment for them to increase one headcount with ai skills we would like to  
[01:28:17] do a little bit more on that and to understand what type of non-listed firms are making this  
[01:28:22] expensive decision investments here is adoption sophistication i would like to you draw your  
[01:28:27] attention to the two only one which is the light blue one you can see first of all the listed firms  
[01:28:33] are um so if it the the the light blue one size is big which means it is more shallow adoption  
[01:28:41] so you can see that the non-listed firms uh in is more is adopting more sophisticately than  
[01:28:47] non-listed firms and then the sophistication is highest among the market service firms  
[01:28:53] yes and similarly we also see this here and then uh let me talk about the pre-ai experience  
[01:29:00] how they matter for generative ai adoption so the dark blue the dark bars are showing  
[01:29:06] ai adopters before 2022 and then the the lighter bars are showing um not ai adopters before 2022  
[01:29:15] you can see that the adoption rate is much higher for that for the uh for those company that has  
[01:29:21] already some ai talent in your company um so this one i only want to highlight one thing that is  
[01:29:29] looking at the left chart where you're looking sorry the right chart when you're looking at  
[01:29:33] the generative ai tools here you can see these are the top occupation the requiring such skills  
[01:29:40] and there are some interesting uh occupation the the light pink one is uh sorry the sorry the  
[01:29:47] brown one the operational specialist so on the left side you sorry the right side you have the  
[01:29:52] ict specialist and a non-ict specialist and one of the occupation that is to me very interesting is  
[01:29:59] operation specialist but they are optimizing the day to day process so that ai can be implemented  
[01:30:05] to speed up the productivities um and uh we also do some regressions to understand what are the  
[01:30:13] firm characteristic that predicts adoption we find larger firms firms in the market service  
[01:30:17] sector and early adopters tend to um tend to be more likely to adopt and then they  
[01:30:23] adopt more intensively we uh we find this evidence for the listed firms and also the non-listed firms  
[01:30:30] so um yes i i'm running out of time and then so i will not read through the slide um and then  

## Discussion (01:30:38 – 01:40:44)

[01:30:38] thank you very much yeah right Christina okay great happy to be here thank you for inviting  
[01:31:04] me to discuss this paper um so i took a flyer that it would uh be um not up to me to explain to you  
[01:31:12] what's going on in the paper so i'm gonna mostly sort of put it in context and and raise some  
[01:31:17] questions i think apply to the paper but in general to this approach to how we measure  
[01:31:23] ai use ai adoption and its implications uh so you know briefly we have an important question  
[01:31:30] about how fast and how unevenly gen ai is diffusing across chinese firms and which firms are early  
[01:31:36] lots and lots and lots of data um so we're doing this very descriptive um approach to using uh keywords  
[01:31:45] it's great to see all the effort to do this in the chinese context and have a bilingual approach  
[01:31:50] to this we see um the you know the headline findings or that we get this 21 of listed firms  
[01:31:56] versus you know less than a percent of private firms by the year end 2025 so very uneven uh we  
[01:32:02] got a sharp jump after the deep-seek introduction um size matters industry matters a lot whether  
[01:32:10] you're a market services or not and these other uh ai uh related tools and this interesting  
[01:32:18] mentions that of specific tools are expanding outside of ict jobs um but the the gen ai the  
[01:32:25] gen ai terms are really kind of going more broadly and so we have this sense that maybe we are looking  
[01:32:32] at different dimensions of ai related activity which i found very exciting um you know i think  
[01:32:38] this is obviously an understudied and very important context i'd like to you know we need more  
[01:32:44] more of this more please um i think the particular that we have some ability to do some  
[01:32:50] um make versus buy distinctions a little bit are we you know taking this off the shelf and  
[01:32:55] using chatbots are we trying to build tools in house what sort of ingredients go into that  
[01:33:00] with the plummeting costs of the chatbots adoption rates you know adoption costs are  
[01:33:05] falling i think that really makes a big difference for where we should see this and how fast um  
[01:33:10] deep seek is a distinct shock from chat gpt this literature loves a shock generated by the  
[01:33:16] availability of a new tool so let's you know do this in more ways and more places and and maybe get  
[01:33:22] some insights that are different from just being in a particular time of year or a particular month  
[01:33:30] we can start to tie this more to a technology and less to the calendar um i think this there's a  
[01:33:36] very interesting critical survey of the literature in section two um i know i have yes i discuss  
[01:33:43] paper because my work is is in this uh discussion of how we've gone about doing this  
[01:33:50] sort of as a field and what the pluses and minuses are and you know everything has its advantages and  
[01:33:56] its disadvantages and there's different ways of doing this and there's a a lot of critical  
[01:34:02] assessment of of what's going on in these different approaches um and it's a firm level  
[01:34:07] approach which i really appreciate so i uh did discuss some papers at the mber measurement  
[01:34:14] conference for ai in may and was at and was kind of banging the job about we need firms we need  
[01:34:20] firms we need firms um so i'm very excited that we have firms here because i was you know  
[01:34:27] kind of pointing to the firming up inequality work in linga bur where we realized that firms  
[01:34:33] are so important for how yeah inequality unfolds in the economy i think we need this for firms i've  
[01:34:40] been working on something for the jep on firming up ai diffusion where we think through how much of  
[01:34:46] what's going on in terms of decisions over job design and complementary investments and other  
[01:34:52] things are happening at the firm level and why we need this firm view to make sense of the  
[01:34:56] phenomenon what goes on inside firms is very complicated and hard to measure and we think  
[01:35:04] about are we looking at just the extensive margin of use we're doing intensive margin that's more  
[01:35:09] deployment what's the breadth of use versus um intensity is something i think we need to know  
[01:35:16] more about what complements are in place and what prerequisites what other technologies  
[01:35:21] were potentially there um there is a lot going on around ai it's really exciting  
[01:35:29] but rather than criticize different approaches to how we would measure this i think we should  
[01:35:35] look at it like the blind men and the elephant that we know very little this is a big complicated  
[01:35:42] beast that we're trying to understand and how we see it depends a lot on what part of the beast  
[01:35:47] we have a hold of and so i think what we really need to do is kind of triangulate on this rather  
[01:35:53] than say one method is going to be the method and i'm a bit worried that if we take section two  
[01:35:57] seriously we're going to actually look like this and we're going to miss out on a lot of really  
[01:36:04] important dimensions of the phenomenon um the other thing i want to emphasize is that adoption  
[01:36:09] rates are fractions and so there's a lot of fights about what's the right rate how do they  
[01:36:14] move i've spent a lot of time trying to sort journalists out all adoption rates are numerators  
[01:36:19] over denominators so a lot of disagreement is about the numerator what are we measuring  
[01:36:25] this is our p and p from this year looking at industrial ai adoption and you can see that  
[01:36:32] you know whether you look at any ai or ai and business functions or specific  
[01:36:37] narrow technical terms and we triangulate in a lot of ways get very very different rates  
[01:36:42] but we're asking about different parts of the elephant the other thing that we're fighting about  
[01:36:47] though is denominators and i think it's really important in the context of this study to keep the  
[01:36:51] denominator straight because we have the listed firms on one hand we have the unlisted firms  
[01:36:58] on the other hand um you know if we're looking at just this middle column here how many chances do  
[01:37:04] you have to cross the extensive margin of ai is fundamentally driven by how many job posts  
[01:37:10] you can post and will post and if we're measuring firm size by posts i think we're measuring the same  
[01:37:16] thing so doing something about the size differential here is really important if we're going to think  
[01:37:23] about it in terms of probabilities of adoption and then if we're going to think about the  
[01:37:28] composition of the labor force then we really have to worry about which jobs we're seeing  
[01:37:33] because i don't think that the manufacturing labor is going to show up so much on the ai on the  
[01:37:39] online postings and so we need to think about what is in that denominator we think about how many firms  
[01:37:45] and when we think about labor per composition um i think we need to think very carefully about  
[01:37:50] what the ai is doing so a lot of us as one desk jockey to a room full of desk jockeys we think  
[01:37:56] about ai on our desk sitting next to us very close to the production process that is very  
[01:38:03] different from what happens in manufacturing firms where technology is often embedded in  
[01:38:08] machines this reminds me when we were studying the rise of icts and we're measuring ict use in  
[01:38:14] manufacturing plants with the number of pcs this is not the right way to do it and what i think  
[01:38:20] is going on is that this industrial ai adoption is actually really showing up in manufacturing  
[01:38:25] through different words not so much chet gbt and deepsick so keeping the context straight and  
[01:38:31] what the ai is doing is very important and then i'm very worried about the problem of yellow bananas  
[01:38:38] the problem of yellow bananas is that when we think of bananas we think of yellow without anyone ever  
[01:38:43] saying the word yellow and what happens uh with diffusion over time um if we think about what is  
[01:38:50] happening in the data generating process of the job posts i'm very worried that routine skills  
[01:38:56] tend to go missing over time there's a whole bunch of citations down at the bottom you can't  
[01:39:00] see where we've looked at job ads and we've seen skills disappear very evolves apparently my  
[01:39:12] spelling does not um and you know i was thinking about what would this look like uh we no longer see  
[01:39:19] jobs for can you use um the internet or email or can you do uh online searches but we do assume  
[01:39:27] all jobs do this and i'm very worried this is going to come very quickly to generative ai  
[01:39:33] job posting data and we'll want to think about that very carefully so i think rather than  
[01:39:38] criticizing each other um you know one of the things that was really hard for me really about a  
[01:39:45] decade ago trying to measure this is the vocabulary changes over time how to call this  
[01:39:51] is really hard and you have all this data for the terminology used so i was thinking we could  
[01:39:57] you know talk to each other about what words we're using in our surveys what words we're  
[01:40:02] finding in the online data out of time so many many suggestions here that i will share with the  
[01:40:16] authors um but a great question rich empirical setting i think we could lean into these gaps  
[01:40:22] to understand more what's happening instead of complaining about whether our measures are  
[01:40:28] the same or not and i'm very excited for the conversation to come so thank you  

## Q&A (01:40:44 – 01:56:54)

[01:40:44] couple quick uh comments uh the yellow banana problem was kind of what i was going to focus  
[01:40:48] on here uh the implication if i look at the numbers is really really small diffusion of ai in china  
[01:40:55] like these numbers are tiny you know below one percent and that makes me really concerned that  
[01:41:00] a lot of this is not being counted in particular seeing that your data is actually named when  
[01:41:12] they're looking for that but that may be a job that uses generative ai doesn't necessarily uh  
[01:41:17] need to include that number and then you know just if you look at the overall diffusion in  
[01:41:23] china you see that the cnn ic says it's like 43 of the population is currently using ai so this number  
[01:41:29] is way smaller than what you show i think it's also interesting that you see gpt as the highest number  
[01:41:35] despite it being blocked in china and not supposed to be used so that's an interesting thing and also  
[01:41:40] that manufacturing is so low 17 compared to others that suggests that what a lot of people  
[01:41:46] talk about is ai turbo charging china's manufacturing and all this might not actually be true  
[01:41:51] which is interesting and the other is that you don't see the soe's adopting any higher actually  
[01:41:56] lower and so a lot of people again also say well china can order the state companies after deep  
[01:42:01] sea came out sasic said adopt this kind of thing and actually you don't see that more so that  
[01:42:06] doesn't suggest a system advantage for china okay um ej uh one of the motivations for organizing  
[01:42:18] this conference was to learn about the ways in which ai use in china is different than the west  
[01:42:30] because we thought that would generate a series of interesting economics questions to understand  
[01:42:38] the motivations for why uh eric um this last six minutes my first second who wrote who wrote  
[01:42:46] an essay on the um the silicon valley consensus and compared it to the beijing consensus and  
[01:42:54] and his point was that there's a very different um attitude to a uh sort of a narrative around  
[01:43:02] the use of machine intelligence uh and his point was that the um beijing consensus was much more  
[01:43:10] industrial and so uh one suggestion is there was a table presented that showed the relative  
[01:43:19] use or the job categories uh that were more common in in listing the use of ai and then in  
[01:43:30] christina survey work and now recently a number of reports have come out from i think all three  
[01:43:35] of the like the uh anthropic open ai and google data on use in different like uh you know they  
[01:43:45] basically do like frequency counts in terms of uh the type of use of their models uh it would be  
[01:43:52] i think you could report on whether there's a meaningful difference in the types of uses  
[01:44:01] so you've already got the data for your paper and you could just do comparative  
[01:44:05] by by mapping to their data uh on whether it's similar or it's being used quite differently and  
[01:44:11] of course all of that will miss christina's point on the right hand side which is probably a first  
[01:44:17] order point we're going to talk about it tomorrow which is the use of ai and control systems for  
[01:44:21] robots so building in some ways on that i mean when i think about innovation systems and sort of the  
[01:44:32] free market system a lot of entrepreneurship you know a lot of information embedded in prices people  
[01:44:37] seeing things differently and making different choices versus a more state driven ai investment  
[01:44:42] i'm just wondering if it's possible to connect this kind of data to that thinking so for example  
[01:44:48] you know in the in the us system you might expect a lot of leading adopters of ai to be  
[01:44:52] entrepreneurs um who are small firms um and are doing a lot of maybe tech hiring in some of  
[01:44:58] the language in this in this paper whereas um and maybe being in widely diverse sectors in the  
[01:45:06] us people trying in all sorts of different places in a very narrow way where you could imagine that in  
[01:45:10] a more centralized system you see heavy adoption and i guess not in the soes but in somewhere  
[01:45:17] that's a bit more concentrated for example which is you know indicating more of a state direction  
[01:45:22] um you know i i think china is somewhere between sort of the old us view and soviet view  
[01:45:29] of innovation and it's a lot closer to the us probably than the soviet one but i don't know  
[01:45:32] i've never seen a sort of test of that and i wonder if this kind of data could help us think  
[01:45:37] about the level of decentralization of innovative ideas yeah just following on that point i'm  
[01:45:44] not sure that that argument is correct actually because what we miss in this story and i'd love  
[01:45:50] to hear both of you um talk about this just a little bit what we miss in the story is this  
[01:45:56] intense competition in china the mark the state signal goes to governments who allocate capital  
[01:46:04] way down on the ground and actually instant fierce fierce competition in some way you  
[01:46:10] you could argue there's first your competition in china right now then in the oligopolous  
[01:46:15] structures that we have in in the united states so there's no reason to expect that we go to soes  
[01:46:22] frankly if we have a picture of intense competition at the bottom and three firms that have 100 survive  
[01:46:31] so does that's a really difficult thing to get at and i take your points which were great what we  
[01:46:36] measure matters and boy it's really hard but i think i think it's a different for me for the larger  
[01:46:43] discussion actually it's an important one for us to understand okay banner christian do you want to  
[01:46:49] follow up on that no okay i had another comment sort of building on this in terms of a jay's framing of  
[01:46:59] trying to understand what's going on in china and this is uh you know this is data about what's  
[01:47:04] going on in china and that's amazing um and there's two core facts i think that come out  
[01:47:12] at least two core facts the first fact is this listed versus nonlisted firms and the rates of  
[01:47:20] mentioning ai in job postings so it's 21 and 1 percent and benchmarked against at least us as i  
[01:47:29] understand that data naming ai and job postings those numbers are low okay now you can say maybe  
[01:47:37] they don't mention the names in china maybe they do but those numbers are low so that was a surprise  
[01:47:41] the second thing which i think is more robust to any complaints about measurement is this change with  
[01:47:47] deep seek and so what is it like why did deep seek uh lead to a change in adoption in china we can all  
[01:47:55] have our hypotheses uh but i think there's two things that are not separated out that are super  
[01:47:59] important one is deep seek is chinese and so maybe because it was chinese and it was local that led  
[01:48:06] to more adoption we know technology adoption in many ways happens to be local the other is it's open  
[01:48:13] and uh is it was it essentially a cost factor that led to deep seek leading to higher adoption  
[01:48:20] in china and i don't know but i don't think we've seen that something that looks like that in the  
[01:48:25] us and so you know accelerated at that time period ostensibly because of the same thing  
[01:48:32] and then so then like there's a bunch of research questions there that i think you might have the  
[01:48:36] data to do uh which is is this because of deep seek and if so is it because of china or openness  
[01:48:49] eventually  
[01:48:51] with the most dramatic thing that's happened in the xp ever which is the rise of agentes  
[01:48:57] so agentes don't have job postings  
[01:49:00] so that that's half of all tokens half of all tokens for ai are not job postings they're  
[01:49:07] they're agentes exactly at no september is 01 you put down january i put down december as deep  
[01:49:19] seek and by march agentes are taking over and they will go from like zero to a hundred to 50  
[01:49:25] percent of the market none of that's in this in these data reasoning models it's a huge change  
[01:49:33] it's a fundamental change in the industry is 01 mini yeah you go from whatever is in the data  
[01:49:39] distribution and it's just pulling and generating based on what's in the training data to actually  
[01:49:44] going out and finding information from you so hallucinations just go down in a massive amount  
[01:49:50] and although 01 had come out not that many people had used it like maybe a few million  
[01:49:54] people in the entire world had used it and then deep seek 01 comes out and part of the part that's  
[01:49:58] interesting is it shows you the full chain of thought so people for the first time realized  
[01:50:02] just how smart these models were beginning to become and that's all confounded with the openness  
[01:50:08] point because china had open weight models before and actually at this point there are open models  
[01:50:12] available from meta that are the leading models in terms of adoption and then deep seek quickly  
[01:50:18] picks up but until this point you have open models so i guess that allows you to  
[01:50:21] disentangle a bit foreign um but not necessarily the other except there was a market signal from  
[01:50:28] the chinese government it says almost at the same time which discouraged entrepreneurs from using  
[01:50:34] anthropic and open ai that so that has an impact that obviously in china as well so  
[01:50:40] all these things coincide so it's really tough to disentangle here do you know how  
[01:50:47] you made the comment about it being banned do you know how effective that was extremely ineffective  
[01:50:53] so uh the one of the funniest things in this space is claud because anthropic tries really hard to  
[01:51:01] make sure that no chinese people can use claud they ban any chinese users they try to stomp out  
[01:51:06] constantly anyone who's trying to use a routing station to do that and then the chinese government  
[01:51:11] has banned claud through the great firewall you can't access it without a vpn but despite the fact  
[01:51:17] that now the chinese government nor anthropic wants chinese people to use claud you have  
[01:51:22] massive adoption of claud code in china through these illicit ways and that's an indication  
[01:51:27] both of the quality of the model and that these controls don't work they just recently  
[01:51:31] it's accurate that uh warning about uh about use of claud from the government in part because  
[01:51:39] of concerns that the data is going to leak to the us but also that when anthropic started to  
[01:51:44] try and poison dis poise and distillation they would kind of downgrade to a worse model or  
[01:51:50] inject information that's going to make it harder to do that kind of research and so the  
[01:51:53] chinese government got freaked out that you know that will be injecting backdoors or something  
[01:51:58] like that or poison data into chinese use of ai do we have evidence of that there's one more  
[01:52:04] piece of that in this answers ben's why how are they using it you can go to the list of chinese  
[01:52:10] models available in china and it will say i'm not sure if it's deep-seek but like jiao mi  
[01:52:14] distilled from lamma distilled from chachi pt distilled from you can get all these distilled  
[01:52:21] these models distilled in a chinese open weight model it's all available ben  
[01:52:30] well i just think the agent agentic point vis-a-vis job postings is so interesting  
[01:52:34] um and that's like a worthwhile thing to study in this data i mean i my impression is that there  
[01:52:40] are not that many zones yet where you just have an agent take over a whole job but you certainly  
[01:52:45] have to take over large parts of what you were doing before particularly encoding but i've also  
[01:52:50] my my related instinct in what i've seen is that some people are much better at using  
[01:52:53] agents than others like people who are like really good at using agents are like super  
[01:52:57] powered uh and it'd be interesting to but so then i might have imagined there would be  
[01:53:00] job postings of the kind like agentic experience um and then you see many fewer job postings like  
[01:53:07] that because you maybe you need fewer people but you want you know i mean i think it'd be very  
[01:53:10] interesting opportunity in uh you know skills for christians as well but like in in us data  
[01:53:14] job postings or revalio or and what's going on the china data to see if you can capture kind  
[01:53:20] of the agentic moment in a shift in language whether before the yellow banana problem is  
[01:53:24] ripe or whatever the right metaphor is there um or too over non-ripe or whatever and because  
[01:53:29] i think that's a fascinating a fascinating linkage yeah so for deep-sea this is very important  
[01:53:36] you know the the shock so uh so one i i tend to agree i think it's uh you know it the the reasoning  
[01:53:43] actually is pretty important the cop you know the uh the chain of thought um so that that's  
[01:53:48] the first time that most people realize okay so it can reason and you know i can i can  
[01:53:53] actually check you know how all models think so i think that that really helps um you know the  
[01:53:58] the adoption rate to go up in china but uh but so interestingly so if we look at deep-sea so  
[01:54:10] so it is to use deep-sea because you know they don't have enough compute so actually that was  
[01:54:15] that moment like after you have this deep-sea deep-sea shock and then people start to realize oh  
[01:54:21] actually look at china there are more than you know deep-sea there are kimmy there are  
[01:54:26] glm there are all kinds of china open source models and these chinese open source models  
[01:54:30] actually uh they are quite good because um i think prior to prior to deep-sea kimmy glm  
[01:54:37] they are actually quite popular but within quite a small circle like within you know for for investors  
[01:54:42] that we are quite familiar with them and for you know some knowledge workers they are you know  
[01:54:46] they are they think that okay these models are are good for use but for general public i think  
[01:54:50] you know especially i think down to down to the level of small media enterprises when they  
[01:54:55] choose to okay so what kind of a large-scale model should i use i probably you know would not  
[01:54:59] think of chinese models but once deep-sea comes you know to be uh so you know the shock it's mentally  
[01:55:14] seems like change uh you know at the mindset a lot of chinese enterprises out there so i think  
[01:55:19] it's not just because deep-sea itself is quite good but to our people like open source models  
[01:55:24] okay really good chinese models really good so we should adopt the use of that yeah great okay  
[01:55:29] you have uh about a minute to respond um just big picture summary yeah so thank you very much  
[01:55:37] actually a lot of the discussion is touching upon uh things we are currently doing so for instance  
[01:55:42] we really try to identify the deep-sea movement what is driving it what is the implication of it  
[01:55:47] and uh another thing is that we also you know the uh so the earlier ai skills were a by-product  
[01:55:54] of the project but actually they are it has a lot of information so if you're looking at you know  
[01:55:59] the the generative ai dot drop-sharing rate is low but the all ai it is about 50 you can see that  
[01:56:13] we're there shifting they are shifting from like a robotic kind of technology technologies  
[01:56:18] towards vision-sensoring and then towards generative ai where they were kind of supporting  
[01:56:24] certain auto work that i should not specify here so basically by looking at how the skills  
[01:56:31] that are evolving in the the company's job posting you can see the technical uh technical  
[01:56:37] advancement they are doing in which direction that is very um that's exactly the something we would  
[01:56:43] like to document a little bit later uh through the data analysis here yeah okay um and thank you  
[01:56:49] very much for all the great ideas that um prompts us to write more papers about this thank you