Study programme 2023-2024Français
First-Order Methods for Large Scale Machine Learning
Programme component of Master's in Computer Engineering and Management (MONS) (day schedule) à la Faculty of Engineering

CodeTypeHead of UE Department’s
contact details
Teacher(s)
UI-M2-IRIGIG-204-MCompulsory UEGILLIS NicolasF151 - Mathématique et Recherche opérationnelle
  • GILLIS Nicolas

Language
of instruction
Language
of assessment
HT(*) HTPE(*) HTPS(*) HR(*) HD(*) CreditsWeighting Term
  • Anglais
Anglais81600022.002nd term

AA CodeTeaching Activity (AA) HT(*) HTPE(*) HTPS(*) HR(*) HD(*) Term Weighting
I-MARO-303First-Order Methods for Large Scale Machine Learning612000Q275.00%
I-MARO-304First-Order Methods for Large Scale Machine Learning - Complements24000Q225.00%

Programme component

Objectives of Programme's Learning Outcomes

  • Imagine, design, develop, and implement conceptual models and computer solutions to address complex problems including decision-making, optimisation, management and production as part of a business innovation approach by integrating changing needs, contexts and issues (technical, economic, societal, ethical and environmental).
    • On the basis of modelling, design a system or a strategy addressing the problem raised; evaluate them in light of various parameters of the specifications.
  • Mobilise a structured set of scientific knowledge and skills and specialised techniques in order to carry out computer and management engineering missions, using their expertise and adaptability.
    • Master and appropriately mobilise knowledge, models, methods and techniques specific to computer management engineering.
    • Assess the validity of models and results in view of the state of science and characteristics of the problem.
  • Plan, manage and lead projects in view of their objectives, resources and constraints, ensuring the quality of activities and deliverables.
    • Define and align the project in view of its objectives, resources and constraints.
  • Work effectively in teams, develop leadership, and make decisions in multidisciplinary, multicultural and international contexts.
    • Interact effectively with others to carry out common projects in various contexts (multidisciplinary, multicultural, and international).
  • Communicate and exchange information in a structured way - orally, graphically and in writing, in French and in one or more other languages - scientifically, culturally, technically and interpersonally, by adapting to the intended purpose and the relevant public.
    • Argue to and persuade customers, teachers and boards, both orally and in writing.
  • Contribute by researching the innovative solution of a problem in engineering sciences.
    • Construct a theoretical or conceptual reference framework, formulate innovative solutions from the analysis of scientific literature, particularly in new or emerging disciplines.
    • Develop and implement conceptual analysis, numerical modelling, software implementations, experimental studies and behavioural analysis.
    • Adequately interpret results taking into account the reference framework within which the research was developed.
    • Communicate, in writing and orally, on the approach and its results in highlighting both the scientific criteria of the research conducted and the theoretical and technical innovation potential, as well as possible non-technical issues.

Learning Outcomes of UE

This class serves as an introduction to first-order methods to tackle large-scale ooptimization problems that arise for example in data analysis and machine learning. A particular attention will be given to stochastic gradient methods that are used to compute the weights in deep neural networks. 

UE Content: description and pedagogical relevance

Organization of the calss:  - Introduction and motivation to the use of firt-order methods.  - Optimal first-order methods for convex optimization.  - Stochastic gradient methods.  - Project: Comparison of first-order methods for solving a classification problem. 

Prior Experience

Non-linear optimization

Type of Teaching Activity/Activities

AAType of Teaching Activity/Activities
I-MARO-303
  • Cours magistraux
  • Travaux pratiques
  • Projet sur ordinateur
I-MARO-304
  • Travaux pratiques

Mode of delivery

AAMode of delivery
I-MARO-303
  • Face-to-face
I-MARO-304
  • Face-to-face

Required Learning Resources/Tools

AARequired Learning Resources/Tools
I-MARO-303Slides and other references available on Moodle
I-MARO-304Slides and other references available on Moodle

Recommended Learning Resources/Tools

AARecommended Learning Resources/Tools
I-MARO-303Bottou, L., Curtis, F. E., & Nocedal, J. (2018). Optimization methods for large-scale machine learning. Siam Review60(2), 223-311.  Newton, D., Yousefian, F., & Pasupathy, R. (2018). Stochastic Gradient Descent: Recent Trends. In Recent Advances in Optimization and Modeling of Contemporary Problems (pp. 193-220). INFORMS.
I-MARO-304Bottou, L., Curtis, F. E., & Nocedal, J. (2018). Optimization methods for large-scale machine learning. Siam Review60(2), 223-311.  Newton, D., Yousefian, F., & Pasupathy, R. (2018). Stochastic Gradient Descent: Recent Trends. In Recent Advances in Optimization and Modeling of Contemporary Problems (pp. 193-220). INFORMS.

Other Recommended Reading

AAOther Recommended Reading
I-MARO-303Not applicable
I-MARO-304Not applicable

Grade Deferrals of AAs from one year to the next

AAGrade Deferrals of AAs from one year to the next
I-MARO-303Unauthorized
I-MARO-304Authorized

Term 2 Assessment - type

AAType(s) and mode(s) of Q2 assessment
I-MARO-303
  • Written examination - Remote
I-MARO-304
  • Written examination - Remote

Term 2 Assessment - comments

AATerm 2 Assessment - comments
I-MARO-303This AA is evaluated via the project
I-MARO-304This AA is evaluated via the project

Term 3 Assessment - type

AAType(s) and mode(s) of Q3 assessment
I-MARO-303
  • Written examination - Remote
I-MARO-304
  • Written examination - Remote

Term 3 Assessment - comments

AATerm 3 Assessment - comments
I-MARO-303idem Q1
I-MARO-304idem Q1
(*) HT : Hours of theory - HTPE : Hours of in-class exercices - HTPS : hours of practical work - HD : HMiscellaneous time - HR : Hours of remedial classes. - Per. (Period), Y=Year, Q1=1st term et Q2=2nd term
Date de dernière mise à jour de la fiche ECTS par l'enseignant : 23/03/2023
Date de dernière génération automatique de la page : 27/04/2024
20, place du Parc, B7000 Mons - Belgique
Tél: +32 (0)65 373111
Courriel: info.mons@umons.ac.be