• AIPressRoom
  • Posts
  • Google at ICML 2023 – Google Analysis Weblog

Google at ICML 2023 – Google Analysis Weblog

Teams throughout Google actively pursue analysis within the discipline of machine studying (ML), starting from principle and software. We construct ML techniques to resolve deep scientific and engineering challenges in areas of language, music, visible processing, algorithm growth, and extra. We purpose to construct a extra collaborative ecosystem with the broader ML analysis group by open-sourcing instruments and datasets, publishing our work, and actively collaborating in conferences.

Google is proud to be a Diamond Sponsor of the fortieth International Conference on Machine Learning (ICML 2023), a premier annual convention, which is being held this week in Honolulu, Hawaii. As a pacesetter in ML analysis, Google has a robust presence at this 12 months’s convention with over 120 accepted papers and lively involvement in plenty of workshops and tutorials. Google can be proud to be a Platinum Sponsor for each the LatinX in AI and Women in Machine Learning workshops. We sit up for sharing a few of our in depth ML analysis and increasing our partnership with the broader ML analysis group.

Registered for ICML 2023? We hope you’ll go to the Google sales space to study extra in regards to the thrilling work, creativity, and enjoyable that goes into fixing a portion of the sphere’s most fascinating challenges. Go to the @GoogleAI Twitter account to seek out out about Google sales space actions (e.g., demos and Q&A periods). See Google DeepMind’s blog to find out about their technical participation at ICML 2023.

Have a look beneath to study extra in regards to the Google analysis being offered at ICML 2023 (Google affiliations in daring).

Scaling Vision Transformers to 22 Billion Parameters (see blog post)Mostafa Dehghani, Josip Djolonga, Basil Mustafa, Piotr Padlewski, Jonathan Heek, Justin Gilmer, Andreas Steiner, Mathilde Caron, Robert Geirhos, Ibrahim Alabdulmohsin, Rodolphe Jenatton, Lucas Beyer, Michael Tschannen, Anurag Arnab, Xiao Wang, Carlos Riquelme, Matthias Minderer, Joan Puigcerver, Utku Evci, Manoj Kumar, Sjoerd van Steenkiste, Gamaleldin F. Elsayed, Aravindh Mahendran, Fisher Yu, Avital Oliver, Fantine Huot, Jasmijn Bastings, Mark Patrick Collier, Alexey Gritsenko, Vighnesh Birodkar, Cristina Vasconcelos, Yi Tay, Thomas Mensink, Alexander Kolesnikov, Filip Pavetić, Dustin Tran, Thomas Kipf, Mario Lučić, Xiaohua Zhai, Daniel Keysers, Jeremiah Harmsen, Neil Houlsby

Fast Inference from Transformers via Speculative DecodingYaniv Leviathan, Matan Kalman, Yossi Matias

 Best of Both Worlds Policy OptimizationChristoph Dann, Chen-Yu Wei, Julian Zimmert

Inflow, Outflow, and Reciprocity in Machine LearningMukund Sundararajan, Walid Krichene

 Transformers Learn In-Context by Gradient DescentJohannes von Oswald, Eyvind Niklasson, Ettore Randazzo, João Sacramento, Alexander Mordvintsev, Andrey Zhmoginov, Max Vladymyrov

Arithmetic Sampling: Parallel Diverse Decoding for Large Language ModelsLuke Vilnis, Yury Zemlyanskiy, Patrick Murray*, Alexandre Passos*, Sumit Sanghai

Differentially Private Hierarchical Clustering with Provable Approximation Guarantees (see blog post)Jacob Imola*, Alessandro Epasto, Mohammad Mahdian, Vincent Cohen-Addad, Vahab Mirrokni

Multi-Epoch Matrix Factorization Mechanisms for Private Machine LearningChristopher A. Choquette-Choo, H. Brendan McMahan, Keith Rush, Abhradeep Thakurta

Simplex Random FeaturesIsaac Reid, Krzysztof Choromanski, Valerii Likhosherstov, Adrian Weller

Pix2Struct: Screenshot Parsing as Pretraining for Visual Language UnderstandingKenton Lee, Mandar Joshi, Iulia Turc, Hexiang Hu, Fangyu Liu, Julian Eisenschlos, Urvashi Khandelwal, Peter Shaw, Ming-Wei Chang, Kristina Toutanova

Mu2SLAM: Multitask, Multilingual Speech and Language ModelsYong Cheng, Yu Zhang, Melvin Johnson, Wolfgang Macherey, Ankur Bapna

Robust Budget Pacing with a Single SampleSantiago Balseiro, Rachitesh Kumar*, Vahab Mirrokni, Balasubramanian Sivan, Di Wang

A Statistical Perspective on Retrieval-Based ModelsSoumya Basu, Ankit Singh Rawat, Manzil Zaheer

Approximately Optimal Core Shapes for Tensor DecompositionsMehrdad Ghadiri, Matthew Fahrbach, Gang Fu, Vahab Mirrokni

Efficient List-Decodable Regression Using BatchesAbhimanyu Das, Ayush Jain*, Weihao Kong, Rajat Sen

Efficient Training of Language Models Using Few-Shot LearningSashank J. Reddi, Sobhan Miryoosefi, Stefani Karp, Shankar Krishnan, Satyen Kale, Seungyeon Kim, Sanjiv Kumar

Fully Dynamic Submodular Maximization Over MatroidsPaul Duetting, Federico Fusco, Silvio Lattanzi, Ashkan Norouzi-Fard, Morteza Zadimoghaddam

GFlowNet-EM for Learning Compositional Latent Variable ModelsEdward J Hu, Nikolay Malkin, Moksh Jain, Katie Everett, Alexandros Graikos, Yoshua Bengio

 Large Language Models Struggle to Learn Long-Tail KnowledgeNikhil Kandpal, Haikang Deng, Adam Roberts, Eric Wallace, Colin Raffel

 Multi-channel Autobidding with Budget and ROI ConstraintsYuan Deng, Negin Golrezaei, Patrick Jaillet, Jason Cheuk Nam Liang, Vahab Mirrokni

 On User-Level Private Convex OptimizationBadih Ghazi, Pritish Kamath, Ravi Kumar, Raghu Meka, Pasin Manurangsi, Chiyuan Zhang

 PAC Generalization via Invariant RepresentationsAdvait U Parulekar, Karthikeyan Shanmugam, Sanjay Shakkottai

 Regularization and Variance-Weighted Regression Achieves Minimax Optimality in Linear MDPs: Theory and PracticeToshinori Kitamura, Tadashi Kozuno, Yunhao Tang, Nino Vieillard, Michal Valko, Wenhao Yang, Jincheng Mei, Pierre Menard, Mohammad Gheshlaghi Azar, Remi Munos, Olivier Pietquin, Matthieu Geist,Csaba Szepesvari, Wataru Kumagai, Yutaka Matsuo

 Speeding Up Bellman Ford via Minimum Violation PermutationsSilvio Lattanzi, Ola Svensson, Sergei Vassilvitskii

 Statistical Indistinguishability of Learning AlgorithmsAlkis Kalavasis, Amin Karbasi, Shay Moran, Grigoris Velegkas

 Test-Time Adaptation with Slot-Centric ModelsMihir Prabhudesai, Anirudh Goyal, Sujoy Paul, Sjoerd van Steenkiste, Mehdi S. M. Sajjadi, Gaurav Aggarwal, Thomas Kipf, Deepak Pathak, Katerina Fragkiadaki>

 Algorithms for Bounding Contribution for Histogram Estimation Under User-Level PrivacyYuhan Liu*, Ananda Theertha Suresh, Wennan Zhu, Peter Kairouz, Marco Gruteser

 Bandit Online Linear Optimization with Hints and QueriesAditya Bhaskara, Ashok Cutkosky, Ravi Kumar, Manish Purohit

 CLUTR: Curriculum Learning via Unsupervised Task Representation LearningAbdus Salam Azad, Izzeddin Gur, Jasper Emhoff, Nathaniel Alexis, Aleksandra Faust, Pieter Abbeel, Ion Stoica

 CSP: Self-Supervised Contrastive Spatial Pre-training for Geospatial-Visual RepresentationsGengchen Mai, Ni Lao, Yutong He, Jiaming Tune, Stefano Ermon

 Ewald-Based Long-Range Message Passing for Molecular GraphsArthur Kosmala, Johannes Gasteiger, Nicholas Gao, Stephan Günnemann

 Fast (1+ε)-Approximation Algorithms for Binary Matrix FactorizationAmeya Velingker, Maximilian Vötsch, David Woodruff, Samson Zhou

 Federated Linear Contextual Bandits with User-Level Differential PrivacyRuiquan Huang, Huanyu Zhang, Luca Melis, Milan Shen, Meisam Hejazinia, Jing Yang

 Investigating the Role of Model-Based Learning in Exploration and TransferJacob C Walker, Eszter Vértes, Yazhe Li, Gabriel Dulac-Arnold, Ankesh Anand, Theophane Weber, Jessica B Hamrick

 Label Differential Privacy and Private Training Data ReleaseRobert Busa-Fekete, Andres Munoz, Umar Syed, Sergei Vassilvitskii

 Lifelong Language Pretraining with Distribution-Specialized ExpertsWuyang Chen*, Yanqi Zhou, Nan Du, Yanping Huang, James Laudon, Zhifeng Chen, Claire Cui

 Multi-User Reinforcement Learning with Low Rank RewardsDheeraj Mysore Nagaraj, Suhas S Kowshik, Naman Agarwal, Praneeth Netrapalli, Prateek Jain

 Multi-View Masked World Models for Visual Robotic ManipulationYounggyo Search engine marketing, Junsu Kim, Stephen James, Kimin Lee, Jinwoo Shin, Pieter Abbeel

 PaLM-E: An Embodied Multimodal Language Model (see blog post)Danny Driess, Fei Xia, Mehdi S. M. Sajjadi, Corey Lynch, Aakanksha Chowdhery, Brian Ichter,Ayzaan Wahid, Jonathan Tompson, Quan Vuong, Tianhe Yu, Wenlong Huang, Yevgen Chebotar, Pierre Sermanet, Daniel Duckworth, Sergey Levine, Vincent Vanhoucke, Karol Hausman, Marc Toussaint, Klaus Greff, Andy Zeng, Igor Mordatch, Pete Florence

 Private Federated Learning with Autotuned CompressionEnayat Ullah*, Christopher A. Choquette-Choo, Peter Kairouz, Sewoong Oh

 Refined Regret for Adversarial MDPs with Linear Function ApproximationYan Dai, Haipeng Luo, Chen-Yu Wei, Julian Zimmert

 Scaling Up Dataset Distillation to ImageNet-1K with Constant MemoryJustin Cui, Ruoche Wan, Si Si, Cho-Jui Hsieh

 The Statistical Benefits of Quantile Temporal-Difference Learning for Value EstimationMark Rowland, Yunhao Tang, Clare Lyle, Rémi Munos, Marc G. Bellemare, Will Dabney

 Unveiling The Mask of Position-Information Pattern Through the Mist of Image FeaturesChieh Hubert Lin, Hung-Yu Tseng, Hsin-Ying Lee, Maneesh Kumar Singh, Ming-Hsuan Yang

 A Simple Zero-Shot Prompt Weighting Technique to Improve Prompt Ensembling in Text-Image ModelsJames Urquhart Allingham*, Jie Ren, Michael W Dusenberry, Xiuye Gu, Yin Cui, Dustin Tran, Jeremiah Zhe Liu, Balaji Lakshminarayanan

 Can Large Language Models Reason About Program Invariants?Kexin Pei, David Bieber, Kensen Shi, Charles Sutton, Pengcheng Yin

 Concurrent Shuffle Differential Privacy Under Continual ObservationJay Tenenbaum, Haim Kaplan, Yishay Mansour, Uri Stemmer

 Fairness in Streaming Submodular Maximization Over a Matroid ConstraintMarwa El Halabi, Federico Fusco, Ashkan Norouzi-Fard, Jakab Tardos, Jakub Tarnawski

 The Flan Collection: Designing Data and Methods for Effective Instruction Tuning (see blog post)Shayne Longpre, Le Hou, Tu Vu, Albert Webson, Hyung Gained Chung, Yi Tay, Denny Zhou, Quoc V Le, Barret Zoph, Jason Wei, Adam Roberts

 Graph Reinforcement Learning for Network Control via Bi-level OptimizationDaniele Gammelli, James Harrison, Kaidi Yang, Marco Pavone, Filipe Rodrigues, Francisco C. Pereira

 Learning-Augmented Private Algorithms for Multiple Quantile ReleaseMikhail Khodak*, Kareem Amin, Travis Dick, Sergei Vassilvitskii

 LegendreTron: Uprising Proper Multiclass Loss LearningKevin H Lam, Christian Walder, Spiridon Penev, Richard Nock

 Measuring the Impact of Programming Language DistributionGabriel Orlanski*, Kefan Xiao, Xavier Garcia, Jeffrey Hui, Joshua Howland, Jonathan Malmaud, Jacob Austin, Rishabh Singh, Michele Catasta*

 Multi-task Differential Privacy Under Distribution SkewWalid Krichene, Prateek Jain, Shuang Tune, Mukund Sundararajan, Abhradeep Thakurta, Li Zhang

 Muse: Text-to-Image Generation via Masked Generative TransformersHuiwen Chang, Han Zhang, Jarred Barber, AJ Maschinot, José Lezama, Lu Jiang, Ming-Hsuan Yang, Kevin Murphy, William T. Freeman, Michael Rubinstein, Yuanzhen Li, Dilip Krishnan

 On the Convergence of Federated Averaging with Cyclic Client ParticipationYae Jee Cho, Pranay Sharma, Gauri Joshi, Zheng Xu, Satyen Kale, Tong Zhang

 Out-of-Domain Robustness via Targeted AugmentationsIrena Gao, Shiori Sagawa, Pang Wei Koh, Tatsunori Hashimoto, Percy Liang

 Polynomial Time and Private Learning of Unbounded Gaussian Mixture ModelsJamil Arbas, Hassan Ashtiani, Christopher Liaw

 Pre-computed Memory or On-the-Fly Encoding? A Hybrid Approach to Retrieval Augmentation Makes the Most of Your ComputeMichiel de Jong, Yury Zemlyanskiy, Nicholas FitzGerald, Joshua Ainslie, Sumit Sanghai, Fei Sha, William W. Cohen

 Scalable Adaptive Computation for Iterative GenerationAllan Jabri*, David J. Fleet, Ting Chen

 Scaling Spherical CNNsCarlos Esteves, Jean-Jacques Slotine, Ameesh Makadia

 STEP: Learning N:M Structured Sparsity Masks from Scratch with PreconditionYucheng Lu, Shivani Agrawal, Suvinay Subramanian, Oleg Rybakov, Christopher De Sa, Amir Yazdanbakhsh

 Stratified Adversarial Robustness with RejectionJiefeng Chen, Jayaram Raghuram, Jihye Choi, Xi Wu, Yingyu Liang, Somesh Jha

 When Does Privileged information Explain Away Label Noise?Guillermo Ortiz-Jimenez*, Mark Collier, Anant Nawalgaria, Alexander D’Amour, Jesse Berent, Rodolphe Jenatton, Effrosyni Kokiopoulou

 Adaptive Computation with Elastic Input SequenceFuzhao Xue*, Valerii Likhosherstov, Anurag Arnab, Neil Houlsby, Mostafa Dehghani, Yang You

 Can Neural Network Memorization Be Localized?Pratyush Maini, Michael C. Mozer, Hanie Sedghi, Zachary C. Lipton, J. Zico Kolter, Chiyuan Zhang

 Controllability-Aware Unsupervised Skill DiscoverySeohong Park, Kimin Lee, Youngwoon Lee, Pieter Abbeel

 Federated Heavy Hitter Recovery Under Linear SketchingAdria Gascon, Peter Kairouz, Ziteng Solar, Ananda Theertha Suresh

 Graph Generative Model for Benchmarking Graph Neural NetworksMinji Yoon, Yue Wu, John Palowitch, Bryan Perozzi, Russ Salakhutdinov

 Invariant Slot Attention: Object Discovery with Slot-Centric Reference FramesOndrej Biza*, Sjoerd van Steenkiste, Mehdi S. M. Sajjadi, Gamaleldin Fathy Elsayed, Aravindh Mahendran, Thomas Kipf

 Multi-task Off-Policy Learning from Bandit FeedbackJoey Hong, Branislav Kveton, Manzil Zaheer, Sumeet Katariya, Mohammad Ghavamzadeh

 Optimal No-Regret Learning for One-Sided Lipschitz FunctionsPaul Duetting, Guru Guruganesh, Jon Schneider, Joshua Ruizhi Wang

 Policy Mirror Ascent for Efficient and Independent Learning in Mean Field GamesBatuhan Yardim, Semih Cayci, Matthieu Geist, Niao He

 Regret Minimization and Convergence to Equilibria in General-Sum Markov GamesLiad Erez, Tal Lancewicki, Uri Sherman, Tomer Koren, Yishay Mansour

 Reinforcement Learning Can Be More Efficient with Multiple RewardsChristoph Dann, Yishay Mansour, Mehryar Mohri

 Reinforcement Learning with History-Dependent Dynamic ContextsMan Tennenholtz, Nadav Merlis, Lior Shani, Martin Mladenov, Craig Boutlier

 User-Defined Event Sampling and Uncertainty Quantification in Diffusion Models for Physical Dynamical SystemsMarc Anton Finzi*, Anudhyan Boral, Andrew Gordon Wilson, Fei Sha, Leonardo Zepeda-Nunez

 Discrete Key-Value BottleneckFrederik Träuble, Anirudh Goyal, Nasim Rahaman, Michael Curtis Mozer, Kenji Kawaguchi, Yoshua Bengio, Bernhard Schölkopf

 DSGD-CECA: Decentralized SGD with Communication-Optimal Exact Consensus AlgorithmLisang Ding, Kexin Jin, Bicheng Ying, Kun Yuan, Wotao Yin

 Exphormer: Sparse Transformers for GraphsHamed Shirzad, Ameya Velingker, Balaji Venkatachalam, Danica J. Sutherland, Ali Kemal Sinop

 Fast, Differentiable and Sparse Top-k: A Convex Analysis PerspectiveMichael Eli Sander*, Joan Puigcerver, Josip Djolonga, Gabriel Peyré, Mathieu Blondel

 Improved Policy Evaluation for Randomized Trials of Algorithmic Resource AllocationAditya Mate, Bryan Wilder, Aparna Taneja, Milind Tambe

 In Search for a Generalizable Method for Source Free Domain AdaptationMalik Boudiaf*, Tom Denton, Bart van Merrienboer, Vincent Dumoulin, Eleni Triantafillou

 Learning Rate Schedules in the Presence of Distribution ShiftMatthew Fahrbach, Adel Javanmard, Vahab Mirrokni, Pratik Worah

 Not All Semantics Are Created Equal: Contrastive Self-Supervised Learning with Automatic Temperature IndividualizationZi-Hao Qiu, Quanqi Hu, Zhuoning Yuan, Denny Zhou, Lijun Zhang, Tianbao Yang

 On the Relationship Between Explanation and Prediction: A Causal ViewAmir-Hossein Karimi*, Krikamol Muandet, Simon Kornblith, Bernhard Schölkopf, Been Kim

 On the Role of Attention in Prompt-TuningSamet Oymak, Ankit Singh Rawat, Mahdi Soltanolkotabi, Christos Thrampoulidis

 PLay: Parametrically Conditioned Layout Generation Using Latent DiffusionChin-Yi Cheng, Forrest Huang, Gang Li, Yang Li

 The Power of Learned Locally Linear Models for Nonlinear Policy OptimizationDaniel Pfrommer, Max Simchowitz, Tyler Westenbroek, Nikolai Matni, Stephen Tu

 Relevant Walk Search for Explaining Graph Neural NetworksPing Xiong, Thomas Schnake, Michael Gastegger, Grégoire Montavon, Klaus Robert Muller,Shinichi Nakajima

 Repository-Level Prompt Generation for Large Language Models of CodeDisha Shrivastava, Hugo Larochelle, Daniel Tarlow

 Robust and Private Stochastic Linear BanditsVasileios Charisopoulos*, Hossein Esfandiari, Vahab Mirrokni

 Simple Diffusion: End-to-End Diffusion for High Resolution ImagesEmiel Hoogeboom, Jonathan Heek, Tim Salimans

 Tied-Augment: Controlling Representation Similarity Improves Data AugmentationEmirhan Kurtulus, Zichao Li, Yann Dauphin, Ekin D. Cubuk

 Why Is Public Pre-Training Necessary for Private Model Training?Arun Ganesh, Mahdi Haghifam*, Milad Nasr, Sewoong Oh, Thomas Steinke, Om Thakkar, Abhradeep Guha Thakurta, Lun Wang

 A Connection Between One-Step RL and Critic Regularization in Reinforcement LearningBenjamin Eysenbach, Matthieu Geist, Sergey Levine, Ruslan Salakhutdinov

 Beyond Uniform Lipschitz Condition in Differentially Private OptimizationRudrajit Das*, Satyen Kale, Zheng Xu, Tong Zhang, Sujay Sanghavi

 Efficient Graph Field Integrators Meet Point CloudsKrzysztof Choromanski, Arijit Sehanobish, Han Lin, Yunfan Zhao, Eli Berger, Tetiana Parshakova, Alvin Pan, David Watkins, Tianyi Zhang, Valerii Likhosherstov, Somnath Basu Roy Chowdhury, Avinava Dubey, Deepali Jain, Tamas Sarlos, Snigdha Chaturvedi, Adrian Weller

 Fast as CHITA: Neural Network Pruning with Combinatorial OptimizationRiade Benbaki, Wenyu Chen, Xiang Meng, Hussein Hazimeh, Natalia Ponomareva, Zhe Zhao, Rahul Mazumder

 Jump-Start Reinforcement Learning (see blog post)Ikechukwu Uchendu*, Ted Xiao, Yao Lu, Banghua Zhu, Mengyuan Yan, Joséphine Simon, Matthew Bennice, Chuyuan Fu, Cong Ma, Jiantao Jiao, Sergey Levine, Karol Hausman

 Learning in POMDPs is Sample-Efficient with Hindsight ObservabilityJonathan Lee, Alekh Agarwal, Christoph Dann, Tong Zhang

 Masked Trajectory Models for Prediction, Representation, and ControlPhilipp Wu, Arjun Majumdar, Kevin Stone, Yixin Lin, Igor Mordatch, Pieter Abbeel, Aravind Rajeswaran

 Predictive Flows for Faster Ford-FulkersonSami Davies, Benjamin Moseley, Sergei Vassilvitskii, Yuyan Wang

 Scaling Laws for Multilingual Neural Machine TranslationPatrick Fernandes, Behrooz Ghorbani, Xavier Garcia, Markus Freitag, Orhan Firat

 Sequential Monte Carlo Learning for Time Series Structure DiscoveryFeras Saad, Brian Patton, Matthew Douglas Hoffman, Rif A. Saurous, Vikash Mansinghka

 Stochastic Gradient Succeeds for BanditsJincheng Mei, Zixin Zhong, Bo Dai, Alekh Agarwal, Csaba Szepesvari, Dale Schuurmans

 Subset-Based Instance Optimality in Private EstimationTravis Dick, Alex Kulesza, Ziteng Solar, Ananda Theertha Suresh

 The Unreasonable Effectiveness of Few-Shot Learning for Machine TranslationXavier Garcia, Yamini Bansal, Colin Cherry, George Foster, Maxim Krikun, Melvin Johnson, Orhan Firat