Abstract
- The focus of artificial intelligence (AI) research is rapidly moving beyond screen-based generative models toward physical AI, an umbrella paradigm for computational systems with physical embodiments that can perceive, reason, and act directly in the real world. In robotics, physical AI is increasingly being implemented through large behavior models and vision-language-action (VLA) models, which learn continuous motor policies from large-scale demonstration datasets. This transition coincides with a structural crisis in tertiary academic medical centers: chronic shortages of residents and skilled surgical assistants that increase the cognitive and physical burden on operating surgeons and may threaten patient safety. This review examines the core engineering mechanisms and clinical applicability of physical AI-driven surgical assistant robots as a practical response to this workforce gap. The engineering foundation of these robots rests on three pillars: VLA models that integrate multimodal perception and commands within a single end-to-end network, online-learning frameworks that adapt in real time to patient-specific tissue variation, and predictive world models that anticipate surgical workflow. Unlike conventional surgeon-controlled robotic systems that only reproduce a surgeon's hand movements, these platforms function as context-aware intelligent partners, with concrete capabilities such as autonomous adaptive retraction, active suction, and autonomous instrument delivery that may reduce tissue injury and preserve surgical workflow continuity. Realizing this potential will require lower real-time control latency, adaptive fail-safe architectures, and resolution of the legal and ethical questions raised by autonomous manipulation. Collaboration among clinicians, engineers, and industry partners will be essential for addressing these challenges and strengthening competitiveness in the global medical AI industry.
-
Keywords: Large behavior model; Physical artificial intelligence; Robotics; Vision-language-action model; Workforce shortage
Introduction
- Recent advances in artificial intelligence (AI) have driven a paradigm shift across medicine. Clinical AI initially centered on diagnostic classification and predictive analytics; the arrival of large language models and screen-confined generative AI subsequently expanded its reach into documentation, medical education, and diagnostic support [1,2]. Yet these models remain passive systems bounded by a digital screen: capable of processing information, but not of acting on the physical world.
- The frontier of robotics and AI research is now converging on physical AI (embodied AI)—an overarching paradigm that couples cognitive intelligence with a robotic embodiment to perceive, reason, and act within unstructured physical environments [3]. Within this paradigm, large behavior models (LBMs) and vision-language-action (VLA) models have emerged as the primary technical frameworks: large-scale deep neural networks trained on extensive behavioral and demonstration datasets that map multimodal perceptual tokens directly to continuous robot control trajectories [4,5]. In the operating room, this evolution redefines the robot's role: from a passive, teleoperated mechanical tool into an active, context-aware assistant that interprets the surgical field and collaborates with the operating surgeon.
- This technological shift coincides with a severe structural crisis in tertiary academic medical centers. The chronic shortage of residents—exacerbated by duty-hour restrictions, evolving trainee preferences, and shifting healthcare policies—compounded by high turnover rates among skilled scrub nurses and surgical assistants, has created an unprecedented operational deficit in operating suites [6,7]. When qualified first or second assistants are scarce or inexperienced, the primary surgeon must repeatedly interrupt the core operative flow to readjust retraction, clear the visual field via suction, or manage instrument exchanges. Empirical studies confirm that surgical team instability and assistant inexperience significantly prolong operative duration, induce surgeon physical and mental exhaustion, and correlate with increased rates of accidental tissue trauma and surgical site complications [8,9].
- Conventional surgical robotic platforms, exemplified by the da Vinci system, have transformed minimally invasive surgery through enhanced dexterity, three-dimensional visualization, and tremor filtration. However, they operate strictly within a surgeon-controlled robotic configuration: they possess no independent environmental awareness and require continuous human control of every arm, offering no relief for the underlying workforce shortage [10].
- This review therefore examines physical AI-driven surgical assistant robots as a practical response to this clinical workforce gap. We outline the foundational engineering frameworks—VLA models, online-learning architectures, and predictive world models—map these frameworks onto concrete clinical applications such as autonomous retraction, active suction, and instrument delivery, and examine the engineering, regulatory, and liability questions that must be resolved before physical AI can transition from preclinical validation into routine surgical practice.
Methods
- A narrative review of the literature was conducted using PubMed/MEDLINE, Scopus, Web of Science, and Google Scholar. The systematic query specifically targeted the rapid emergence of generative physical AI, LBMs, and VLA robotic frameworks published between January 2023 and July 2026. The search utilized combinations of Medical Subject Headings (MeSH) terms and free-text keywords: (“Physical AI” OR “Large Behavior Model” OR “Vision-Language-Action” OR “VLA model”) AND (“surgical robotics” OR “autonomous surgery” OR “surgical assistant” OR “workflow recognition”) AND (“workforce shortage” OR “patient safety” OR “burnout”). In addition, foundational historical papers on surgical autonomy and seminal workforce shortage reports published prior to 2023 were identified through manual citation tracking and included to provide necessary historical and contextual grounding. Studies limited to standard teleoperated systems without autonomous cognitive processing or context-aware feedback were excluded. The retrieved literature was synthesized to characterize current engineering frameworks and evaluate their practical relevance to the operating-room workforce shortage.
Applications in surgical robotics: core engineering mechanisms
- 1. VLA models in the surgical field
- Traditional surgical robotic architectures rely on a modular pipeline: separate software modules process endoscopic video (computer vision), parse verbal or manual commands (user interface), and compute kinematic trajectories for the robotic joints (motion control). This fragmented design accumulates computational latency at every hand-off, and an error or artifact introduced at the vision stage can propagate downstream into inappropriate mechanical behavior.
- Physical AI bypasses this bottleneck through VLA models: a unified, end-to-end neural architecture that encodes multimodal input—real-time stereoscopic endoscopic video, the spatial coordinates of surgical instruments, and the surgeon's verbal or gestural cues—into a single token space and maps it directly onto low-level joint torque and velocity commands. Robotic Transformer 2 (RT-2) first demonstrated, in general robotic manipulation, that internet-scale vision-language data could be transferred directly to robot control [4]; more recently, dual-system foundation models such as NVIDIA's GR00T N1 have paired a vision-language reasoning module with a diffusion-based action module to pursue both contextual reasoning and fast, precise motion generation within a single humanoid-robot framework [5]. Applied to surgery, this architecture allows the system to interpret the spatial proximity of an energy device alongside the surgeon’s spoken intent and adjust its own pose accordingly, without an intermediate hand-coded control layer.
- 2. “Learning while deploying” through online learning
- A central challenge in surgical automation is the deformable, idiosyncratic nature of living tissue. Unlike the rigid, predictable geometries of industrial manufacturing, the operative field varies enormously between patients, and tissue itself behaves viscoelastically—deforming, shifting, and changing structurally in response to manipulation, hydration, or bleeding. A static, “frozen” model trained only on pre-recorded data will inevitably struggle when it encounters anatomy or tissue compliance it has not seen before.
- To address this limitation, physical AI research has begun to explore “learning while deploying”: robots that continue to refine their control policy using data gathered from the live clinical environment itself. A hierarchical, language-conditioned imitation-learning framework has recently demonstrated autonomous execution of discrete steps of cholecystectomy despite inter-specimen tissue variability [11], and a reinforcement-learning policy trained in an open-source laparoscopic simulation environment has been shown to transfer to physical hardware in a zero-shot manner [12]. Together, these studies suggest that a robot can use haptic and torque feedback from its end-effector, together with real-time visual input, to estimate the viscoelasticity and friction of an individual patient's tissue and adjust its grasp and trajectory accordingly—though translating this adaptability into real-time, safety-validated intraoperative use remains an active area of investigation.
- 3. Surgical workflow recognition and predictive world models
- For a robotic platform to act as an effective assistant, it must track where the operation stands within the overall procedure—a capability known as surgical workflow recognition. Deep semantic segmentation networks identify and track anatomical structures (vessel boundaries, nerve pathways, tumor margins) and active instruments within the surgical field, and recent work has extended this recognition beyond coarse surgical phases to fine-grained procedural steps [13].
- Building on this foundation, predictive world models add a further layer of anticipation: given the current state of the surgical field and the surgeon's recent instrument trajectory, a world model estimates the state that is likely to follow, and recent work has explored learning such robot policies directly from surgical video [14]. If the system recognizes, for example, that dissection of a vascular pedicle has just been completed, it can anticipate that ligation or clip application will follow next, and pre-position the camera or instrument arm accordingly—helping preserve a continuous, uninterrupted surgical flow.
Comparative analysis and clinical translation
- 1. Surgeon-controlled robotic systems vs. physical AI collaboration
- Understanding the clinical value of physical AI requires comparing it with the surgeon-controlled robotic paradigm that dominates current robotic surgery. Surgical robot autonomy has been described along a spectrum—from systems that simply replicate the surgeon's motion, through those that execute discrete subtasks independently, to those capable of planning execution toward a higher-level goal [10]. Table 1 summarizes the resulting technical and functional distinctions.
- 2. Automation of core intraoperative tasks
- The practical value of physical AI in addressing the assistant shortage is best illustrated through three intraoperative tasks that traditionally require dedicated personnel (Fig. 1).
- • Autonomous adaptive retraction: Tissue retraction is essential but physically demanding, typically delegated to junior residents or assistants. Ineffective or unsteady retraction directly impairs visual exposure and can induce tissue ischemia or parenchymal tearing [15]. Preclinical investigations utilizing phantoms and porcine models have demonstrated that autonomous robotic retraction systems—governed by visual-tactile feedback loops—can continuously track tissue boundaries and apply optimal, force-regulated grasping tensions [15]. As tissue relaxes or shifts with respiration, the system dynamically recalibrates its spatial positioning. While highly validated on benchtop and animal platforms, translation into human in vivo surgery represents an active, near-term clinical goal currently undergoing rigorous safety verification.
- • Active visual field cleansing (autonomous suction): Sudden microvascular bleeding can obscure the operative field within seconds, forcing the surgeon to halt dissection to direct an assistant or manage suction personally. Recent engineering frameworks have demonstrated autonomous blood-pooling detection and suction trajectory planning in phantom and ex vivo models [16]. Utilizing deep convolutional networks, the system identifies blood accumulation as a distinct visual phase shift, computes the coordinates of the bleeding focus, and calculates collision-free paths avoiding critical neural and vascular structures. While fully autonomous suction remains a prospective clinical application in human operating suites, these preclinical milestones highlight its viability for unburdening operating surgeons during acute bleeding events.
- • Autonomous instrument delivery (robotic scrub assistant): Instrument hand-off between scrub personnel and the surgeon depends on precise, well-rehearsed coordination that staff turnover routinely disrupts. A VLA-based robotic scrub assistant has recently been shown to interpret a surgeon’s spoken instrument request or open-palm gesture, scan the instrument tray, identify and grasp the requested tool, and deliver it into the surgeon’s hand in the correct orientation—all within a single learned policy [17]. Such systems may reduce the delays and hand-off errors associated with inexperienced scrub staff.
- 3. Clinical evaluation across surgical subspecialties
- The clinical translation of physical AI must be tailored to the distinct anatomical and mechanical demands of each surgical discipline. Table 2 summarizes representative applications and their anticipated clinical impact across three major fields.
Discussion
- 1. Engineering bottlenecks and safety architecture
- Despite this clinical promise, several engineering and regulatory barriers must be resolved before physical AI can be deployed widely. Chief among the technical limitations is computational latency: processing high-resolution stereoscopic video into semantic segmentation maps, running a predictive world-model simulation, and generating torque commands all demand substantial computation. Even a brief latency spike exceeding a few milliseconds can leave the robot’s actions out of sync with a rapidly changing surgical field—particularly during acute hemorrhage, when a delayed response applied to an outdated anatomical position could cause tissue laceration [18].
- Autonomous decision-making also calls for a different class of safety architecture than conventional industrial automation, where a rigid emergency stop is often sufficient. In surgery, an abrupt mechanical freeze during active retraction or vascular compression can itself be dangerous. The field will need adaptive fail-safe mechanisms, grounded in explainable AI, that allow a system encountering high uncertainty—an anatomical anomaly outside its training distribution, for example—to degrade gracefully rather than freeze or fail outright: releasing tension, returning to a neutral position, and alerting the surgeon through haptic or auditory cues to resume manual control.
- Regulatory bodies such as Korea’s Ministry of Food and Drug Safety and the US Food and Drug Administration do not yet have standardized frameworks for evaluating devices that continue to update their own control policies after deployment through online learning. Developing validation guidelines that are rigorous enough to ensure safety, yet flexible enough to accommodate a continually evolving model, remains an open institutional challenge [19].
- 2. Legal liability and ethical considerations
- The shift from passive tool to autonomous agent raises new questions of medical liability. Under the conventional teleoperative paradigm, liability is relatively straightforward: because the robot functions purely as an extension of the surgeon’s hand, any resulting tissue injury is generally attributed to surgeon error. Once a physical AI system independently plans and executes part of a procedure—an autonomous suction pathway, for instance, or a change in grasping force during retraction—responsibility for an unexpected complication becomes harder to assign among the operating surgeon, the institution, and the device manufacturer. Was the complication due to a surgeon's failure to supervise adequately, a flaw in the algorithm's training data, or a fault in the hospital's network infrastructure? Legal and product-liability frameworks will need to be updated to define accountability clearly according to the level of autonomy the system was exercising at the time of the incident—protecting patients without discouraging continued innovation [20].
- 3. Building an interdisciplinary medical-engineering ecosystem
- Resolving these challenges will require moving away from the traditional, one-directional model in which engineers build a platform and clinicians subsequently adopt it. Designing a surgical world model that performs reliably in practice depends on the continuous input of clinicians who best understand real-world unmet needs and anatomical nuance, from the earliest stages of development; conversely, clinicians will need to develop sufficient technical literacy to understand the operating boundaries and probabilistic nature of these models [21]. Structured, interdisciplinary centers—where computer vision researchers, roboticists, and surgical subspecialists work side by side, and where clinical observations and algorithmic refinement circulate through a continuous feedback loop—offer the most realistic path toward a physical AI system that is safe, clinically grounded, and genuinely effective in addressing the operating-room workforce shortage.
Conclusions
- Physical AI marks a substantive step in the evolution of computer-assisted intervention, moving the surgical robot from a passive mechanical tool toward a context-aware partner in the operating room. By unifying perception and action through end-to-end VLA architectures and predictive world models, and by automating demanding but routine tasks such as retraction, suction, and instrument handling, these platforms offer a concrete engineering response to the shortage of residents and surgical assistants straining academic medical centers.
- Realizing this potential in practice will depend on resolving two categories of challenge in parallel: an engineering challenge—minimizing real-time control latency and standardizing adaptive fail-safe architectures—and an institutional one—establishing clear legal and ethical accountability for autonomous manipulation. Neither can be solved in isolation; both call for a genuinely collaborative medical-engineering-industry ecosystem, built around clinicians who understand the unmet needs of the operating room and engineers who can translate those needs into the next generation of surgical foundation models. Establishing this collaborative structure now is not only the most direct route to easing the current operating-room workforce crisis, but also a realistic starting point for securing a meaningful position in the global medical AI and surgical robotics market.
Article information
-
Conflicts of interest
No potential conflict of interest relevant to this article was reported.
-
Funding
None.
-
Author contributions
All the work was done by Eui-Suk Sung.
Fig. 1.Architectural schematic of a physical artificial intelligence (AI)-driven surgical assistant platform. The closed-loop system integrates multimodal sensory inputs, including endoscopic vision, surgeon gestures and voice commands, and force-torque feedback, into a unified VLA network paired with predictive world models. The model adapts to patient-specific tissue deformation through online reinforcement learning (“learning while deploying”) and executes core physical tasks, including autonomous retraction, active suction, and scrub delivery. An integrated fail-safe layer monitors uncertainty and initiates graceful degradation or manual takeover when uncertainty exceeds prespecified safety boundaries. 3D, three-dimensional.
Table 1.Technical and functional comparison of robotic paradigms
|
Evaluation category |
Traditional surgeon-controlled robotic systems |
Next-generation physical artificial intelligence assistants |
|
Control architecture |
Continuous 1:1 teleoperation that mechanically replicates the surgeon's input. |
Autonomous, context-aware path planning driven by an integrated cognitive model. |
|
Operational role |
A passive tool that extends the surgeon's reach and precision. |
An active assistant that interprets context and collaborates with the operating surgeon. |
|
Contextual awareness |
None; all situational judgment rests with the surgeon. |
High; VLA and world models track anatomy and workflow transitions in real time. |
|
Workforce impact |
Requires a dedicated human operator for every active arm; does not relieve the assistant shortage. |
Helps offset the shortage of residents and scrub assistants by automating routine tasks. |
|
Adaptability |
Bounded by fixed, preprogrammed kinematics. |
Continually refines its control policy through online learning. |
Table 2.Specialty-specific implementation of physical AI and anticipated clinical benefits
|
Surgical domain |
Limitations of current systems |
Physical AI application |
Anticipated clinical benefits |
|
Abdominal and hepatobiliary/colorectal surgery (e.g., hepatopancreatobiliary resection, total mesorectal excision) |
Multi-organ access requires repeated manual clutching to reposition assistant arms, interrupting surgical flow and increasing the risk of instrument collision across quadrants. |
World-model-based path prediction anticipates instrument trajectories to avoid multi-arm collisions, while force-controlled autonomous retraction follows the dissection plane. |
May reduce risks associated with manual repositioning and contribute to shorter operative time; autonomous control of grasping pressure may help prevent ischemic parenchymal injury. |
|
Otolaryngology-head and neck surgery (e.g., laryngeal microsurgery, transoral surgery, thyroidectomy, neck dissection) |
Extremely confined three-dimensional corridors make assistant suction and retraction skill central to procedural feasibility and create a high risk of catastrophic neurovascular compromise. |
A VLA model jointly processes endoscopic vision and surgeon intent to execute micro-retraction and active suction while systematically tracking and avoiding vital structures, including the recurrent laryngeal nerve, superior laryngeal nerve, lingual nerve, hypoglossal nerve, marginal mandibular nerve, internal carotid artery, and internal jugular vein. |
May lower the risk of permanent vocal cord paralysis, cranial nerve deficits, and major hemorrhage and reduce physical interference from human assistants in narrow transoral or cervical spaces. |
|
Cardiothoracic and urological surgery (e.g., minimally invasive lobectomy, robot-assisted radical prostatectomy) |
Cardiac and respiratory motion, combined with deep and delicate neurovascular anatomy, makes real-time synchronization difficult for conventional control schemes. |
An online-learning policy adapts to the patient's respiratory cycle and organ motion, allowing the robot to continuously track a dynamically moving tissue surface. |
May help reduce bleeding risk during fine vascular dissection and support faster postoperative recovery through enhanced nerve preservation. |
References
- 1. Mesko B. The ChatGPT (generative artificial intelligence) revolution has made artificial intelligence approachable for medical professionals. J Med Internet Res 2023;25:e48392.ArticlePubMedPMC
- 2. Thirunavukarasu AJ, Ting DS, Elangovan K, Gutierrez L, Tan TF, Ting DS. Large language models in medicine. Nat Med 2023;29:1930–40.ArticlePubMedPDF
- 3. Yang GZ, Bellingham J, Dupont PE, Fischer P, Floridi L, Full R, et al. The grand challenges of Science Robotics. Sci Robot 2018;3:eaar7650.ArticlePubMed
- 4. Brohan A, Brown N, Carbajal J, Chebotar Y, Chen X, Choromanski K, et al. RT-2: vision-language-action models transfer web knowledge to robotic control. arXiv [Preprint] 2023;[cited 2026 Sep 17]. https://doi.org/10.48550/arXiv.2307.15818Article
- 5. Bjorck J, Castaneda F, Cherniadev N, Da X, Ding R, Fan L, et al. GR00T N1: an open foundation model for generalist humanoid robots. arXiv [Preprint] 2025;[cited 2026 Sep 17]. https://doi.org/10.48550/arXiv.2503.14734Article
- 6. Oslock WM, Satiani B, Way DP, Tamer RM, Maurer J, Hawley JD, et al. A contemporary reassessment of the US surgical workforce through 2050 predicts continued shortages and increased productivity demands. Am J Surg 2022;223:28–35.ArticlePubMed
- 7. Dedeilia A, Sotiropoulos MG, Hanrahan JG, Janga D, Dedeilias P, Sideris M. Medical and surgical education challenges and innovations in the COVID-19 era: a systematic review. In Vivo 2020;34:1603–11.ArticlePubMedPMC
- 8. Kurmann A, Keller S, Tschan-Semmer F, Seelandt J, Semmer NK, Candinas D, et al. Impact of team familiarity in the operating room on surgical complications. World J Surg 2014;38:3047–52.ArticlePubMedPDF
- 9. Stefanidis D, Wang F, Korndorffer JR, Dunne JB, Scott DJ. Robotic assistance improves intracorporeal suturing performance and safety in the operating room while decreasing operator workload. Surg Endosc 2010;24:377–82.ArticlePubMedPDF
- 10. Attanasio A, Scaglioni B, De Momi E, Fiorini P, Valdastri P. Autonomy in surgical robotics. Annu Rev Control Robot Auton Syst 2021;4:651–79.Article
- 11. Kim JW, Chen JT, Hansen P, Shi LX, Goldenberg A, Schmidgall S, et al. SRT-H: a hierarchical framework for autonomous surgery via language-conditioned imitation learning. Sci Robot 2025;10:eadt5254.ArticlePubMed
- 12. Long Y, Lin A, Kwok DH, Zhang L, Yang Z, Shi K, et al. Surgical embodied intelligence for generalized task autonomy in laparoscopic robot-assisted surgery. Sci Robot 2025;10:eadt3093.ArticlePubMed
- 13. Demir KC, Schieber H, Weise T, Roth D, May M, Maier A, et al. Deep learning in surgical workflow analysis: a review of phase and step recognition. IEEE J Biomed Health Inform 2023;27:5405–17.ArticlePubMed
- 14. He Y, Guo P, Xu M, Li Z, Myronenko A, Imans D, et al. Cosmos-H-Surgical: learning surgical robot policies from videos via world modeling. arXiv [Preprint] 2025;[cited 2026 Sep 17]. https://doi.org/10.48550/arXiv.2512.23162Article
- 15. Shi Y, Mo H, Xiao X, Fang J, Li L, Min K, et al. A safety-enhanced multi-modal objectives motion fusion method for autonomous retraction in robotic surgery. IEEE Trans Autom Sci Eng 2025;22:15990–6000.Article
- 16. Shin C, Ferguson PW, Pedram SA, Ma J, Dutson EP, Rosen J. Autonomous tissue manipulation via surgical robot using learning based model predictive control. In: ICRA 2019: Proceedings of the 2019 International Conference on Robotics and Automation; 2019 May 20-24; Montreal, Canada. IEEE; 2019. p. 3875-81.Article
- 17. Li S, Wang J, Dai R, Ma W, Ng WY, Hu Y, et al. RoboNurse-VLA: robotic scrub nurse system based on vision-language-action model. In: IROS 2025: Proceedings of the 2025 IEEE/RSJ International Conference on Intelligent Robots and Systems; 2025 Oct 19-25; Hangzhou, China. IEEE; 2025. p. 3986-93.Article
- 18. Hashimoto DA, Rosman G, Rus D, Meireles OR. Artificial intelligence in surgery: promises and perils. Ann Surg 2018;268:70–6.ArticlePubMedPMC
- 19. Muehlematter UJ, Daniore P, Vokinger KN. Approval of artificial intelligence and machine learning-based medical devices in the USA and Europe (2015-20): a comparative analysis. Lancet Digit Health 2021;3:e195–203.ArticlePubMed
- 20. Ong JC, Chang SY, William W, Butte AJ, Shah NH, Chew LS, et al. Ethical and regulatory challenges of large language models in medicine. Lancet Digit Health 2024;6:e428–32.ArticlePubMed
- 21. Topol EJ. High-performance medicine: the convergence of human and artificial intelligence. Nat Med 2019;25:44–56.ArticlePubMedPDF
Citations
Citations to this article as recorded by
