Publications

• O. Surinta, L. Schomaker and M.A. Wiering. Handwritten Character Classification Using the Hotspot Feature Extraction Technqiue, International Conference on Pattern Recognition Applications and Methods (ICPRAM), 2012.
  
  
• H. van Hoof, T. van der Zant, and M.A. Wiering. Adaptive Visual Face Tracking for an Autonomous Robot. BNAIC'11: Belgian Dutch Artificial Intelligence Conference, 2011.
  
  
• A.D. Pietersma, L. Schomaker, and M.A. Wiering. Kernel Learning in Support Vector Machines using Dual-Objective Optimization. BNAIC'11: Belgian Dutch Artificial Intelligence Conference, 2011.
  
  
• H. van Seijen, S. Whiteson, H. van Hasselt, M.A. Wiering. Exploiting Best-Match Equations for Efficient Reinforcement Learning. Journal of Machine Learning Research (JMLR), 12, 2045-2094, 2011.
  
  
• J. de Vries, I. Hooge, M.A. Wiering, F. Verstraten. How longer saccade latencies lead to a competition for salience. Psychological Science, 2011.
  
  
• A. Shantia, E. Begue, M.A. Wiering. Connectionist Reinforcement Learning for Intelligent Unit Micro Management in StarCraft. International Joint Conference on Neural Networks, 2011.
  
  
• J. de Vries, I. Hooge, M.A. Wiering, F. Verstraten. Saccadic selection and crowding in visual search: Stronger lateral masking leads to shorter search times. Experimental Brain Research, 2011.
  
  
• M.A. Wiering, H. van Hasselt, A.D. Pietersma, L. Schomaker. Reinforcement Learning Algorithms for solving Classification Problems. Proceedings of IEEE International Symposium on Approximate Dynamic Programming and Reinforcement Learning (ADPRL), Paris, 2011,
  
  
• A. Abdullah, R.C. Veltkamp and M.A. Wiering. Ensembles of Novel Visual Keywords Descriptors for Image Categorization . ICARV, 2010.
  
  
• M.A. Wiering. Zelflerende verkeerslichtnetwerken, BLIND (interdisciplinair tijdschrift), special issue about networks, 2010.
  
  
• T.P. Schmidt, M.A. Wiering, A.C. van Rossum, R.A.J. van Elburg, T.C. Andringa, B. Valkenier. Robust Real-Time Vowel Classification with an Echo State Network, Workshop on "Cognitive and neural models for automated processing of speech and text" 2010 (CONAS).
  
  
• M.A. Wiering and T. Kooi. Region Enhanced Neural Q-learning for Solving Model-based POMDPs. International Joint Conference on Neural Networks, 2010.
  
  
• M.A. Wiering. Self-Play and Using an Expert to Learn to Play Backgammon with Temporal Difference Learning . Journal of Intelligent Learning Systems and Applications, 2010, 2, pp: 57-68.
  
  
• M.M. Drugan and M.A. Wiering. Feature selection for Bayesian Network Classifiers using the MDL-FS score, International Journal of Approximate Reasoning, Elsevier, 2010.
  
  
• A. Abdullah, R.C. Veltkamp, and M.A. Wiering. Fixed Partitioning and Salient Points with MPEG-7 Cluster Correlograms for Image Categorization. Pattern Recognition, Volume 43. Issue 3, Pages 650-662, March 2010.
  
  
• A. Abdullah, R.C. Veltkamp, and M.A. Wiering. An Ensemble of Deep Support Vector Machines for Image Categorization. Proceedings of the International Conference on Soft Computing and Pattern Recognition (SocPar), pp. 301-306, 2009. BEST PAPER AWARD
  
  
• H. van Hasselt and Marco Wiering. Using Continuous Action Spaces to Solve Discrete Problems. Proceedings of the International Joint Conference on Neural Networks (IJCNN), Atlanta, USA, 2009.
  
  
• A. Abdullah, R.C. Veltkamp, and M.A. Wiering. Spatial Pyramids and Two-layer Stacking SVM classifiers for Image Categorization: A Comparative Study. Proceedings of the International Joint Conference on Neural Networks (IJCNN), Atlanta, USA, 2009.
  
  
• M.A. Wiering and H. van Hasselt. The QV Family Compared to Other Reinforcement Learning Algorithms. Proceedings of IEEE International Symposium on Approximate Dynamic Programming and Reinforcement Learning (ADPRL), Nashville, USA, pp. 101-108, 2009.
  
  
• H. van Seijen, H. van Hasselt, S. Whiteson, and M. Wiering. A Theoretical and Empirical Analysis of Expected Sarsa. In ADPRL 2009: Proceedings of the IEEE Symposium on Adaptive Dynamic Programming and Reinforcement Learning, pp. 177-184, 2009.
  
  
• M.A. Wiering and H. van Hasselt. Ensemble Algorithms in Reinforcement Learning. IEEE Transactions on Systems, Man, and Cybernetics, Part B, Volume 38, 4, 930-936, 2008.
  
  
• M. van Otterlo, M. Wiering, M. Dastani, and J-J. Meyer. A Characterization of Sapient Agents, In R.V. Mayorga and L. Perlovsky (Eds.) Toward Artificial Sapience, Principles and Methods for Wise Systems, pp. 129-141. Berlin: Springer, 2008.
  
  
• R. Opsomer, P. Knoth, F. van Polen, J. Trapman and M.A. Wiering. Categorizing Children: Automated Text Classification of CHILDES files. BNAIC'08: Proceedings of the 20 Belgium-Netherlands Conference on Artificial Intelligence, A. Nijholt, M. Pantic, M. Poel and H. Hondorp (eds.), pp. 209-216, 2008.
  
  
• L. Pape, J. de Gruijl, and M.A. Wiering, 2008. Democratic Liquid State Machines for Music Recognition. In: Speech, Audio, Image and Biomedical Signal Processing using Neural Networks, Bookseries: Studies in Computational Intelligence, vol 83. B. Prasad and S.R.M. Prasanna (Eds.), 2008.
  
  
• L. Lefakis and M.A. Wiering. Semi-Supervised Methods for Handwritten Character Recognition using Active Learning. BNAIC'07: Proceedings of the 19th Belgium-Netherlands Conference on Artificial Intelligence, Mehdi Dastani and Edwin de Jong (eds.), pp. 205-212, 2007.
  
  
• L. Pape, B.G. Ruessink, M.A. Wiering and I.L. Turner. Recurrent Neural Network Modeling of Nearshore Sandbar Behavior. Neural Networks, Special Issue on Earth and Environmental Sciences, 20, 509-518, 2007.
  
  
• Marco Wiering and Edwin D. de Jong. Computing Optimal Stationary Policies for Multi-objective Markov Decision Processes. Proceedings of IEEE International Symposium on Approximate Dynamic Programming and Reinforcement Learning (ADPRL), Honolulu, HI, USA, pp. 158-165, 2007.
  
  
• Azizi Abdullah and Marco Wiering. CIREC: Cluster Correlogram Image Retrieval and Categorization using MPEG-7 Descriptors. Proceedings of IEEE International Symposium on Computational Intelligence in Image and Signal Processing (CIISP), Honolulu, HI, USA, pp. 431-437, 2007.
  
  
• Hado van Hasselt and Marco Wiering. Reinforcement Learning in Continuous Action Spaces. Proceedings of IEEE International Symposium on Approximate Dynamic Programming and Reinforcement Learning (ADPRL), Honolulu, HI, USA, pp. 272-279, 2007.
  
  
• Hado van Hasselt and Marco Wiering. Convergence of Model-Based Temporal Difference Learning for Control. Proceedings of IEEE International Symposium on Approximate Dynamic Programming and Reinforcement Learning (ADPRL), Honolulu, HI, USA, pp. 60-67, 2007.
  
  
• Marco Wiering and Hado van Hasselt. Two Novel On-policy Reinforcement Learning Algorithms based on TD(lambda)-methods. Proceedings of IEEE International Symposium on Approximate Dynamic Programming and Reinforcement Learning (ADPRL), Honolulu, HI, USA, pp. 280-287, 2007.
  
  
• Wilco Moerman, Bram Bakker and Marco Wiering. Hierarchical Assignment of Behaviors to Subpolicies. NIPS'2007 workshop on Hierarchical Organization of Behavior: Computational, Psychological and Neural Perspectives, 2007.
  
  
• Tijn van der Zant, Lambert Schomaker, Marco Wiering, Axel Brink. Cognitive Developmental Pattern Recognition: Learning to Learn Proceedings of the IEEE International Conference on Systems, Man and Cybernetics, pp. 1208-1213, 2006
  
  
• L. Pape, B.G. Ruessink, M.A. Wiering and I.L. Turner, Neural network modeling of nearshore sandbar behavior . Proceedings of the 2006 International Joint Conference on Neural Networks, Vancouver, Canada, pp. 8735-8742, 2006.
  
  
• W. de Back, E.D. de Jong, and M.A. Wiering. Red Queen Dynamics in a predator-prey ecosystem Proceedings of the Genetic and Evolutionary Computation Conference (GECCO-06), Maarten Keijzer et al. editors, pp. 381-382, 2006
  
  
• J. van Diggelen, E.D. de Jong, and M.A. Wiering. Strategies for Ontology Negotiation: Finding the Right Level of Generality International workshop on agent communication, held with AAMAS'06, 2006
  
  
• J.R. de Gruijl and M.A. Wiering. Musical Instrument Classification using Democratic Liquid State Machines Benelearn'06: Proceedings of the 15th Belgian-Dutch Conference on Machine Learning, pp. 33-40, edited by Y. Saeys, E. Tsiporkova, B. De Baets, and Y. Van de Peer, 2006
  
  
• L. Zwanepol Klinkmeijer, E.D. de Jong, and M.A. Wiering. A Serial Population Genetic Algorithm for Dynamic Optimization Problems Benelearn'06: Proceedings of the 15th Belgian-Dutch Conference on Machine Learning, pp. 41-48, edited by Y. Saeys, E. Tsiporkova, B. De Baets, and Y. Van de Peer, 2006
  
  
• M. Wiering, J.P. Patist, and H. Mannen Learning to PLay Board Games using Temporal Difference Methods . Technical Report, Utrecht University, UU-CS-2005-048, 30 pages, 2005.
  
  
• M. Wiering. QV(lambda)-learning: A New On-policy Reinforcement Learning Algorithm. Proceedings of the 7th European Workshop on Reinforcement Learning, D. Leone (editor), pages 17-18, 2005.
  
  
• M. Wiering. Comparing Training Paradigms for Learning to Play Backgammon Proceedings of the 7th European Workshop on Reinforcement Learning, D. Leone (editor), pages 29-30, 2005.
  
  
• T. van der Zant, M. Wiering, and J. van Eijck. On-line robot learning using the interval estimation algorithm Proceedings of the 7th European Workshop on Reinforcement Learning, D. Leone (editor), pages 11-12, 2005.
  
  
• S. Maas, M. Wiering, and B. Verhaar. Reinforcement Learning of a Pneumatic Robot Arm Controller Proceedings of the 7th European Workshop on Reinforcement Learning, D. Leone (editor), pages 23-24, 2005.
  
  
• H. van Kuilenburg, M. Wiering, and M. den Uyl. A Model Based Method for Automatic Facial Expression Recognition. Proceedings of the 16th European Conference on Machine Learning (ECML'05), J. Gama et a. (eds), Springer-Verlag Berlin Heidelberg, pages 194-205,2005.
  
  
• R.R. Negenborn, B. De Schutter, M.A. Wiering, and H. Hellendoorn. Learning-based model predictive control for Markov decision processes. Proceedings of the 16th IFAC World Congress, Prague, Czech Republic, July 2005.
  
  
• M. Wiering, F. Mignogna, and B, Maassen Evolving Neural Networks for Forest Fire Control Benelearn'05: Proceedings of the 14th Belgian-Dutch Conference on Machine Learning, pages 113 - 120, edited by M. van Otterlo, M. Poel, and A. Nijholt, 2005
  
  
• M. Sindlar and M. Wiering. A Modular Approach to Facial Expression Recognition Benelearn'05: Proceedings of the 14th Belgian-Dutch Conference on Machine Learning, pages 81 - 88, edited by M. van Otterlo, M. Poel, and A. Nijholt, 2005
  
  
• R.R. Negenborn, B. De Schutter, M.A. Wiering, and H. Hellendoorn. Learning-based model predictive control for Markov decision processes. Tech. rep. 04-021, Delft Center for Systems and Control, Delft University of Technology, Delft, The Netherlands, Sept. 2004.
  
  
• R.R. Negenborn, B. De Schutter, M.A. Wiering, and J. Hellendoorn, Experience-based model predictive control using reinforcement learning. Proceedings of the 8th TRAIL Congress 2004 - A World of Transport, Infrastructure and Logistics - CD-ROM, Rotterdam, The Netherlands, Nov. 2004.
  
  
• M. Wiering. Convergence and Divergence in Standard and Averaging Reinforcement Learning Proceedings of the 15th European Conference on Machine Learning (ECML'04), edited by J-F Boulicaut, F. Esposito, F. Giannotti, and D. Pedreschi, pp. 477-488, Springer-Verlag Berlin Heidelberg, 2004.
  
  
• M. Wiering, S. Leijnen, A. Koster, S. van Weers, and W. de Back. Autonomous Intelligent Robots at Utrecht University. International Journal of Advanced Robotic Systems, 1(2), pages 125-128, 2004.
  
  
• M. Wiering, J. van Veenen, J. Vreeken, and A. Koopman. Intelligent Traffic Light Control. Technical Report UU-CS-2004-029, University Utrecht, 2004.
  
  
• D. Wierstra and M. Wiering. Utile Distinction Hidden Markov Models. Proceedings of the Twenty-first International Conference on Machine Learning (ICML'04), pp. 855-862, ACM Press, 2004.
  
  
• M. Wiering, J. Vreeken, J. van Veenen, and A. Koopman. Simulation and Optimization of Traffic in a City. IEEE Intelligent Vehicles symposium (IV'04), 2004.
  
  
• H. Mannen and M. Wiering. Learning to play chess using TD(lambda)-learning with database games Benelearn'04: Proceedings of the Thirteenth Belgian-Dutch Conference on Machine Learning, edited by A. Nowe, T. Lenaerts, and K. Steenhout, pp.72-79, 2004
  
  
• J-P. Patist and M. Wiering. Learning to play draughts using temporal difference learning with neural networks and databases Benelearn'04: Proceedings of the Thirteenth Belgian-Dutch Conference on Machine Learning, edited by A. Nowe, T. Lenaerts, and K. Steenhout, pp.87-94, 2004
  
  
• M. Wiering. Memory-based Memetic Algorithms Benelearn'04: Proceedings of the Thirteenth Belgian-Dutch Conference on Machine Learning, edited by A. Nowe, T. Lenaerts, and K. Steenhout, pp.191-198, 2004
  
  
• M. van Otterlo, M. Wiering, M. Dastani, and J-J. Meyer. A Characterization of Sapient Agents, First International Conference on Integration of Knowledge Intensive Multi-Agent Systems (KIMAS-03), edited by H. Hexmoor, IEEE Press, Boston, MA, pages 172-177, 2003.
  
  
• M. Wiering. Hierarchical Mixtures of Naive Bayesian Classifiers. European Conference on Machine Learning (ECML'2003) Workshop on Probabilistic Graphical Models for Classification, edited by P. Larranaga, J.A> Lozano, J.M. Pena, and I. Inza, pages 93-104, 2003.
  
  
• M. Wiering and F. Mignogna. Learning to Control Forest Fires with ESP. Proceedings of the Sixth European Workshop on Reinforcement Learning, edited by Alain Dutech and Olivier Buffet, pp. 22-23, 2003.
  
  
• M. Wiering. Intelligent Traffic Light Control. ERCIM News Special: Cognitive Systems, 53, pp. 40-41, 2003.
  
  
• M. Wiering. Evolving Causal Neural Networks. Benelearn'02: Proceedings of the Twelfth Belgian-Dutch Conference on Machine Learning, edited by Marco Wiering, pp. 103-108, 2002
  
  
• M. Wiering. Hierarchical Mixtures of Naive Bayesian Classifiers. BNAIC'02: Proceedings of the Thirteenth Belgium-Netherlands Conference on Artificial Intelligence, Hendrik Blockeel and Marc Denecker (eds.), pp. 363-370, 2002.
  
  
• M. Wiering. Model-based Reinforcement Learning in Dynamic Environments. Technical Report CS-UU-2002-029, Utrecht University, 2002.
  
  
• S. Reynolds and M. Wiering. Fast Q(lambda) revisited. Technical Report CSRP-02-2, University of Birmingham, School of Computer Science, 2002.
  
  
• M. Wiering. Hierarchical Mixtures of Naive Bayes Classifiers. Technical Report CS-UU-2002-003, Utrecht University, 2002.
  
  
• J. de Jong and M. Wiering. Multiple Ant Colony Systems for the Busstop Allocation Problem. BNAIC'01: Proceedings of the Thirteenth Belgium-Netherlands Conference on Artificial Intelligence, pp. 141-148, 2001.
  
  
• Marco Wiering, R.P. Salustowicz, and Juergen Schmidhuber. Model-based Reinforcement Learning for Evolving Soccer Strategies. In Computational Intelligence in Games, chapter 5. Editors N. Baba and L. Jain. pp. 99-131, 2001.
  
  
• Marco Wiering. Reinforcement Learning in Dynamic Environments using Instantiated Information. Machine Learning: Proceedings of the Eighteenth International Conference (ICML'2001), pp. 585-592, 2001.
  
  
• K. ten Tusscher, S. ten Hagen and M. Wiering The influence of commmunication on the choice to behave cooperatively. Proceedings of the Tenth Belgian-Dutch Conference on Machine Learning. Editor Ad Feelders. pp. 39-46, 2000.
  
  
• Marco Wiering. Multi-Agent Reinforcement Learning for Traffic Light control. Machine Learning: Proceedings of the Seventeenth International Conference (ICML'2000), pp. 1151-1158, 2000.
  
  
• Marco Wiering, Ben Krose, and Frans Groen. Learning in Multi-Agent Systems. Technical Report, University of Amsterdam, 1999.
  
  
• Marco Wiering, R.P. Salustowicz, and Juergen Schmidhuber. Reinforcement Learning Soccer Teams with Incomplete World Models Neural Networks for Robot Learning. Special issue of Autonomous Robots, Vol 7(1), pp. 77-88, 1999.
  
  
• Marco Wiering. Explorations in Efficient Reinforcement Learning. Ph.D. thesis. February 1999 (784K).
  
  
• Marco Wiering, R.P. Salustowicz, and Juergen Schmidhuber. CMAC models learn to play soccer. Proceedings of the 8th International Conference on Artificial Neural Networks (ICANN'98), 443-448. In L. Niklasson and M. Boden and T. Ziemke (eds.), Springer-Verlag, London, 1998.
  
  
• Marco Wiering and Marco Dorigo. Learning to Control Forest Fires. Umweltinformatik'98: Vernetzte Strukturen in Informatik, Umwelt und Wirtschaft, Proceedings des 12. Internationalen Symposiums 'Informatik den Umweltschutz', H.-D. Haasis, K.C. Ranze (eds.), pp 378-388, 1998.
  
  
• Marco Wiering and Juergen Schmidhuber. Efficient Model-Based Exploration. Proceedings of the Fifth International Conference on Simulation of Adaptive Behavior (SAB'98): From Animals to Animats 5, 223-228, R. Pfeiffer, B. Blumberg, J. A. Meyer and S. W. Wilson (eds.), MIT Press/Bradford Books, 1998.
  
  
• Marco Wiering and Juergen Schmidhuber. Learning Exploration Policies with Models. Conference on Automated Learning and Discovery (CONALD'98), 1998.
  
  
• Marco Wiering and Juergen Schmidhuber. Fast Online Q(lambda). Machine Learning, 33(1), 105-115, 1998.
  
  
• Marco Wiering and Juergen Schmidhuber. Speeding Up Q(lambda)-learning . In Proceedings of the Tenth European Conference on Machine Learning (ECML'98), pp. 352-363, 1998. (13 pages).
  
  
• Marco Wiering and Juergen Schmidhuber. HQ-Learning. Adaptive Behavior, 6:2, 219-246, 1997.
  
  
• Rafal Salustowicz, Marco Wiering and Juergen Schmidhuber. Learning Team Strategies: Soccer Case Studies. Machine Learning, 33, (2/3), 1-19, 1998.

You can also just check the abstract or pick up the bibtex entry.


• Rafal Salustowicz, Marco Wiering and Juergen Schmidhuber. Evolving soccer strategies. In N. Kasabov, R. Kozma, K. Ko, R. O'Shea, G. Coghill, and T. Gedeon, editors, Progress in Connectionist-based Information Systems: Proceedings of the Fourth International Conference on Neural Information Processing ICONIP'97, volume 1, 502-505,
,1997 (74K).
You can also just check the abstract or pick up the bibtex entry.


• Rafal Salustowicz, Marco Wiering and Juergen Schmidhuber. On learning soccer strategies. In W. Gerstner, A. Germond, M. Hasler, and J.-D. Nicoud, editors, Proceedings of the Seventh International Conference on Artificial Neural Networks (ICANN'97), volume 1327 of Lecture Notes in Computer Science, 769-774. Springer-Verlag Berlin Heidelberg, 1997 (68K)
  You can also just check the abstract or pick up the bibtex entry.
  
  
• Juergen Schmidhuber, Jieyu Zhao and Marco Wiering. Shifting Inductive Bias with Success-Story Algorithm, Adaptive Levin Search, and Incremental Self-Improvement. Machine Learning, 28:1, 105-130, 1997.
  
  
• Marco Wiering and Juergen Schmidhuber. HQ-Learning: Discovering Markovian Subgoals for non-Markovian Reinforcement Learning. Technical Report IDSIA-95-96, October 1996 (108K).
  
  
• Juergen Schmidhuber and Jieyu Zhao and Marco Wiering. Simple principles of metalearning. Technical Report IDSIA-69-96, June 1996 (195 K).
  
  
• Rafal Salustowicz, Marco Wiering and Juergen Schmidhuber. Learning team strategies with multiple policy-sharing agents: A soccer case study. Technical Report IDSIA-29-97, IDSIA, Lugano, Switzerland. -- 20 pages, 1997 (134K).

You can also just check the abstract or pick up the bibtex entry.


• Marco Wiering and Juergen Schmidhuber. Solving POMDPs with Levin Search and EIRA. Machine Learning: Proceedings of the thirteenth International Conference. 534-542, 1996 (86K).
  
  
• Marco Wiering. TD Learning of Game Evaluation Functions with Hierarchical Neural Architectures. Master's thesis, University of Amsterdam, Holland, April 1995 (241K).
  
  
• Marco Wiering and Ben Kroese. TD Learning of Game Evaluation Functions with Hierarchies of Adaptive Experts. University of Amsterdam, Holland, April 1995 (59K).