The draft report comments -
Introduction
A. whereas from Mary Shelley's Frankenstein's Monster to the classical myth of Pygmalion, through the story of Prague's Golem to the robot of Karel Čapek, who coined the word, people have fantasised about the possibility of building intelligent machines, more often than not androids with human features;
B. whereas now that humankind stands on the threshold of an era when ever more sophisticated robots, bots, androids and other manifestations of artificial intelligence ("AI") seem poised to unleash a new industrial revolution, which is likely to leave no stratum of society untouched, it is vitally important for the legislature to consider all its implications;
C. whereas between 2010 and 2014 the average increase in sales of robots stood at 17% per year and in 2014 sales rose by 29%, the highest year-on-year increase ever, with automotive parts suppliers and the electrical/electronics industry being the main drivers of the growth; whereas annual patent filings for robotics technology have tripled over the last decade;
D. whereas in the short to medium term robotics and AI promise to bring benefits of efficiency and savings, not only in production and commerce, but also in areas such as transport, medical care, education and farming, while making it possible to avoid exposing humans to dangerous conditions, such as those faced when cleaning up toxically polluted sites; whereas in the longer term there is potential for virtually unbounded prosperity;
E. whereas at the same time the development of robotics and AI may result in a large part of the work now done by humans being taken over by robots, so raising concerns about the future of employment and the viability of social security systems if the current basis of taxation is maintained, creating the potential for increased inequality in the distribution of wealth and influence;
F. whereas the causes for concern also include physical safety, for example when a robot's code proves fallible, and the potential consequences of system failure or hacking of connected robots and robotic systems at a time when increasingly autonomous applications come into use or are impending whether it be in relation to cars and drones or to care robots and robots used for maintaining public order and policing;
G. whereas many basic questions of data protection have already become the subject of consideration in the general contexts of the internet and e-commerce, but whereas further aspects of data ownership and the protection of personal data and privacy might still need to be addressed, given that applications and appliances will communicate with each other and with databases without humans intervening or possibly without their even being aware of what is going on;
H. whereas the 'soft impacts' on human dignity may be difficult to estimate, but will still need to be considered if and when robots replace human care and companionship, and whereas questions of human dignity also can arise in the context of 'repairing' or enhancing human beings;
I. whereas ultimately there is a possibility that within the space of a few decades AI could surpass human intellectual capacity in a manner which, if not prepared for, could pose a challenge to humanity's capacity to control its own creation and, consequently, perhaps also to its capacity to be in charge of its own destiny and to ensure the survival of the species;
J. whereas several foreign jurisdictions, such as the US, Japan, China and South Korea, are considering, and to a certain extent have already taken, regulatory action with respect to robotics and AI, and whereas some Member States have also started to reflect on possible legislative changes in order to take account of emerging applications of such technologies;
K. whereas European industry could benefit from a coherent approach to regulation at European level, providing predictable and sufficiently clear conditions under which enterprises could develop applications and plan their business models on a European scale while ensuring that the EU and its Member States maintain control over the regulatory standards to be set, so as not to be forced to adopt and live with standards set by others, that is to say the third states which are also at the forefront of the development of robotics and AI;
General principles
L. whereas, until such time, if ever, that robots become or are made self-aware, Asimov's Laws must be regarded as being directed at the designers, producers and operators of robots, since those laws cannot be converted into machine code;
M. whereas, nevertheless, a series of rules, governing in particular liability and ethics and reflecting the intrinsically European and humanistic values that characterise Europe's contribution to society, are necessary;
N. whereas the European Union could play an essential role in establishing basic ethical principles to be respected in the development, programming and use of robots and AI and in the incorporation of such principles into European regulations and codes of conduct, with the aim of shaping the technological revolution so that it serves humanity and so that the benefits of advanced robotics and AI are broadly shared, while as far as possible avoiding potential pitfalls;
O. whereas a gradualist, pragmatic cautious approach of the type advocated by Jean Monnet should be adopted for Europe;
P. whereas it is appropriate, in view of the stage reached in the development of robotics and AI, to start with civil liability issues and to consider whether a strict liability approach based on who is best placed to insure is not the best starting point;
Liability
Q. whereas, thanks to the impressive technological advances of the last decade, not only are today's robots able to perform activities which used to be typically and exclusively human, but the development of autonomous and cognitive features – e.g. the ability to learn from experience and take independent decisions – has made them more and more similar to agents that interact with their environment and are able to alter it significantly; whereas, in such a context, the legal responsibility arising from a robot’s harmful action becomes a crucial issue;
R. whereas a robot's autonomy can be defined as the ability to take decisions and implement them in the outside world, independently of external control or influence; whereas this autonomy is of a purely technological nature and its degree depends on how sophisticated a robot's interaction with its environment has been designed to be;
S. whereas the more autonomous robots are, the less they can be considered simple tools in the hands of other actors (such as the manufacturer, the owner, the user, etc.); whereas this, in turn, makes the ordinary rules on liability insufficient and calls for new rules which focus on how a machine can be held – partly or entirely – responsible for its acts or omissions; whereas, as a consequence, it becomes more and more urgent to address the fundamental question of whether robots should possess a legal status;
T. whereas, ultimately, robots' autonomy raises the question of their nature in the light of the existing legal categories – of whether they should be regarded as natural persons, legal persons, animals or objects – or whether a new category should be created, with its own specific features and implications as regards the attribution of rights and duties, including liability for damage;
U. whereas under the current legal framework robots cannot be held liable per se for acts or omissions that cause damage to third parties; whereas the existing rules on liability cover cases where the cause of the robot’s act or omission can be traced back to a specific human agent such as the manufacturer, the owner or the user and where that agent could have foreseen and avoided the robot’s harmful behaviour; whereas, in addition, manufacturers, owners or users could be held strictly liable for acts or omissions of a robot if, for example, the robot were categorised as a dangerous object or if it fell within product liability rules;
V. whereas in the scenario where a robot can take autonomous decisions, the traditional rules will not suffice to activate a robot's liability, since they would not make it possible to identify the party responsible for providing compensation and to require this party to make good the damage it has caused;
X. whereas the shortcomings of the current legal framework are apparent in the area of contractual liability insofar as machines designed to choose their counterparts, negotiate contractual terms, conclude contracts and decide whether and how to implement them make the traditional rules inapplicable, which highlights the need for new, more up-to-date ones;
Y. whereas, as regards non-contractual liability, can only cover damage caused by a robot's manufacturing defects and on condition that the injured person is able to prove the actual damage, the defect in the product and the causal relationship between damage and defect (strict liability or liability without fault);
Z. whereas, notwithstanding the scope of the Directive 85/374/EEC, the current legal framework would not be sufficient to cover the damage caused by the new generation of robots, insofar as they can be equipped with adaptive and learning abilities entailing a certain degree of unpredictability in their behaviour, since these robots would autonomously learn from their own, variable experience and interact with their environment in a unique and unforeseeable manner;The report, underGeneral principles concerning the development of robotics and artificial intelligence for civil use, accordingly
1. Calls on the Commission to propose a common European definition of smart autonomous robots and their subcategories by taking into consideration the following characteristics of a smart robot: o acquires autonomy through sensors and/or by exchanging data with its environment (inter - connectivity) and trades and analyses data o is self-learning (optional criterion) o has a physical support; o adapts its behaviours and actions to its environment;
2. Considers that a system of registration of advanced robots should be introduced, and calls on the Commission to establish criteria for the classification of robots with a view to identifying the robots that would need to be registered;
3. Underlines that many robotic applications are still in an experimental phase; welcomes the fact that more and more research projects are being funded with national and European money; calls on the Commission and the Member States to strengthen financial instruments for research projects in robotics and ICT; emphasises that sufficient resources need to be devoted to the search for solutions to the social and ethical challenges that the technological development and its applications raise;
4. Asks the Commission to foster research programmes that include a mechanism for short-term verification of the outcomes in order to understand what real risks and opportunities are associated with the dissemination of these technologies; calls on the Commission to combine all its effort in order to guarantee a smoother transition for these technologies from research to commercialisation on the market;
Ethical principles
5. Notes that the potential for empowerment through the use of robotics is nuanced by a set of tensions or risks relating to human safety, privacy, integrity, dignity, autonomy and data ownership;
6. Considers that a guiding ethical framework for the design, production and use of robots is needed to complement the legal recommendations of the report and the existing national and Union acquis; proposes, in the annex to the resolution, a framework in the form of a charter consisting of a code of conduct for robotics engineers, of a code for research ethics committees when reviewing robotics protocols and of model licences for designers and users;
7. Points out that the guiding ethical framework should be based on the principles of beneficence, non-maleficence and autonomy, as well as on the principles enshrined in the EU Charter of Fundamental Rights, such as human dignity and human rights, equality, justice and equity, non-discrimination and non-stigmatisation, autonomy and individual responsibility, informed consent, privacy and social responsibility, and on existing ethical practices and codes;
A European Agency
8. Calls for the creation of a European Agency for robotics and artificial intelligence in order to provide the technical, ethical and regulatory expertise needed to support the relevant public actors, at both EU and Member State level, in their efforts to ensure a timely and well-informed response to the new opportunities and challenges arising from the technological development of robotics;
9. Considers that the potential of robotics use and the present investment dynamics justify the European Agency being equipped with a proper budget and being staffed with regulators and external technical and ethical experts dedicated to the cross-sectorial and multidisciplinary monitoring of robotics-based applications, identifying standards for best practice, and, where appropriate, recommending regulatory measures, defining new principles and addressing potential consumer protection issues and systematic challenges; asks the Commission and the European Agency to report to the European Parliament on the latest developments in robotics on an annual basis; Intellectual property rights and the flow of data
10. Notes that there are no legal provisions that specifically apply to robotics, but that existing legal regimes and doctrines can be readily applied to robotics while some aspects appear to need specific consideration; calls on the Commission to come forward with a balanced approach to intellectual property rights when applied to hardware and software standards, and codes that protect innovation and at the same time foster innovation; calls on the Commission to elaborate criteria for an ‘own intellectual creation’ for copyrightable works produced by computers or robots;
11. Calls on the Commission and the Member States to ensure that, in the development of any EU policy on robotics, privacy and data protection guarantees are embedded in line with the principles of necessity and proportionality; calls, in this regard, on the Commission to foster the development of standards for the concepts of privacy by design and privacy by default, informed consent and encryption;
12. Points out that the use of personal data as a 'currency' with which services can be 'bought' raises new issues in need of clarification; stresses that the use of personal data as a 'currency' must not lead to a circumvention of the basic principles governing the right to privacy and data protection;
Standardisation, safety and security
13. Calls on the Commission to continue to work on the international harmonisation of technical standards, in particular together with the European Standardisation Organisations and the International Standardisation Organisation, in order to avoid fragmentation of the inter nal market and to meet consumers’ concerns; asks the Commission to analyse existing European legislation with a view to checking the need for adaption in light of the development of robotics and artificial intelligence;
14. Emphasises that testing robots in real-life scenarios is essential for the identification and assessment of the risks they might entail, as well as of their technological development beyond a pure experimental laboratory phase; underlines, in this regard, that testing of robots in real-life scenarios, in particular in cities and on roads, raises numerous problems and requires an effective monitoring mechanism; calls on the Commission to draw up uniform criteria across all Member States which individual Member States should use in order to identify areas where experiments with robots are permitted;
Autonomous vehicles
15. Considers that the automotive sector is in most urgent need of European and global rules to ensure the cross-border development of automated vehicles so as to fully exploit their economic potential and benefit from the positive effects of technological trends; emphasises that fragmented regulatory approaches would hinder implementation and jeopardise European competitiveness; notes that although current private international law rules on traffic accidents applicable within the EU do not need urgent modification to accommodate the development of autonomous vehicles, simplifying the current dual system for defining applicable law (based on Regulation (EC) No 864/2007 of the European Parliament and of the Council and the 1971 Hague Convention on the law applicable to traffic accidents) would improve legal certainty and limit possibilities for forum shopping;
Care robots
16. Points out that human contact is one of the fundamental aspects of human care; believes that replacing the human factor with robots could dehumanise caring practices;
Medical robots
17. Underlines the importance of appropriate training and preparation for doctors and care assistants in order to secure the hig hest degree of professional competence possible, as well as to protect patients' health; underlines the need to define the minimum professional requirements that a surgeon must meet in order to be allowed to use surgical robots; emphasises the special importance of training for users to allow them to familiarise themselves with the technological requirements in this field; draws attention to the rising trend towards self-diagnosis using a mobile robot which makes diagnoses and might take over the role of a doctor;
Human repair and enhancement
18. Notes the great potential of robotics in the field of repairing and compensating for damaged organs and human functions, but also the complex questions raised in particular by the possibilities of human enhancement; asks for the establishment of committees on robot ethics in hospitals and other health care institutions tasked with considering and assisting in resolving unusual, complicated ethical problems involving issues that affect the care and treatment of patients; calls on the Commission and the Member States to develop guidelines to aid in the establishment and functioning of such committees;
Drones (RPAS)
19. Stresses the importance of a European framework for remotely piloted aircraft systems (RPAS) to prote ct the safety, security and privacy of EU citizens, and calls on the Commission for a follow-up to the recommendations of the European Parliament resolution of 29 October 2015 on safe use of remotely piloted aircraft systems (RPAS), known as unmanned aerial vehicles (UAVs), in the field of civil aviation
20. Draws attention to the Commission's forecast that by 2020 Europe might be facing a shortage of up to 825000 ICT professionals and that 90% of jobs will require at least basic digital skills; welcomes the Commission’s initiative of proposing a roadmap for the possible use and revision of a Digital Competence framework and descriptors of Digital Competences for all levels of learners;
21. Considers that getting more young women interested in a digital career and placing more women in digital jobs would benefit the digital industry, women themselves and Europe's economy; calls on the Commission and the Member States to launch initiatives in order to support women in ICT and to boost their e-skills;
22. Calls on the Commission to start monitoring job trends more closely, with a special focus on the creation and loss of jobs in the different fields/areas of qualification in order to know in which fields jobs are being created and those in which jobs are being destroyed as a result of the increased use of robots;
23. Bearing in mind the effects that the development and deployment of robotics and AI might have on employment and, consequently, on the viability of the social security systems of the Member States, consideration should be given to the possible need to introduce corporate reporting requirements on the extent and proportion of the contribution of robotics and AI to the economic results of a company for the purpose of taxation and social security contributions; takes the view that in the light of the possible effects on the labour market of robotics and AI a general basic income should be seriously considered, and invites all Member States to do so;
Liability
24. Considers that robots' civil liability is a crucial issue which needs to be addressed at EU level so as to ensure the same degree of transparency, consistency and legal certainty throughout the European Union for the benefit of consumers and businesses alike;
25. Asks the Commission to submit, on the basis of Article 114 of the Treaty on the Functioning of the European Union, a proposal for a legislative instrument on legal questions related to the development of robotics and artificial intelligence foreseeable in the next 10-15 years, following the detailed recommendations set out in the annex hereto; further calls on the Commission, once technological developments allow the possibility for robots whose degree of autonomy is higher than what is reasonably predictable at present to be developed, to propose an update of the relevant legislation in due time;
26. Considers that, whatever legal solution it applies to robots' liability in cases other than those of damage to property, the future legislative instrument should in no way restrict the type or the extent of the damages which may be recovered, nor should it limit the forms of compensation which may be offered to the aggrieved party, on the sole grounds that damage is caused by a non-human agent;
27. Considers that the future legislative instrument should provide for the application of strict liability as a rule, thus requiring only proof that damage has occurred and the establishment of a causal link between the harmful behaviour of the robot and the damage suffered by the injured party;
28. Considers that, in principle, once the ultimately responsible parties have been identified, their liability would be proportionate to the actual level of instructions given to the robot and of its autonomy, so that the greater a robot's learning capability or autonomy is, the lower other parties' responsibility should be, and the longer a robot's 'education' has lasted, the greater the responsibility of its 'teacher' should be; notes, in particular, that skills resulting from 'education' given to a robot should be not confused with skills depending strictly on its self-learning abilities when seeking to identify the person to whom the robot's harmful behaviour is actually due;
29. Points out that a possible solution to the complexity of allocating responsibility for damage caused by increasingly autonomous robots could be an obligatory insurance scheme, as is already the case, for instance, with cars; notes, nevertheless, that unlike the insurance system for road traffic, where the insurance covers human acts and failures, an insurance system for robotics could be based on the obligation of the producer to take out an insurance for the autonomous robots it produces;
30. Considers that, as is the case with the insurance of motor vehicles, such an insurance system could be supplemented by a fund in order to ensure that reparation can be made for damage in cases where no insurance cover exists; calls on the insurance industry to develop new products that are in line with the advances in robotics;
31. Calls on the Commission, when carrying out an impact assessment of its future legislative instrument, to explore the implications of all possible legal solutions, such as:
a) establishing a compulsory insurance scheme whereby, similarly to what already happens with cars, producers or owners of robots would be required to take out insurance cover for the damage potentially caused by their robots;
b) ensuring that a compensation fund would not only serve the purpose of guaranteeing compensation if the damage caused by a robot was not covered by an insurance – which would in any case remain its primary goal – but also that of allowing various financial operations in the interests of the robot, such as investments, donations or payments made to smart autonomous robots for their services, which could be transferred to the fund;
c) allowing the manufacturer, the programmer, the owner or the user to benefit from limited liability insofar as smart autonomous robots would be endowed with a compensation fund – to which all parties could contribute in varying proportions – and damage to property could only be claimed for within the limits of that fund, other types of damage not being subject to such limits;
d) deciding whether to create a general fund for all smart autonomous robots or to create an individual fund for each and every robot category, and whether a contribution should be paid as a one-off fee when placing the robot on the market or whether periodic contributions should be paid during the lifetime of the robot;
e) ensuring that the link between a robot and its fund would be made visible by an individual registration number appearing in a specific EU register, which would allow anyone interacting with the robot to be informed about the nature of the fund, the limits of its liability in case of damage to property, the names and the functions of the contributors and all other relevant details;
f) creating a specific legal status for robots, so that at least the most sophisticated autonomous robots could be established as having the status of electronic persons with specific rights and obligations, including that of making good any damage they may cause, and applying electronic personality to cases where robots make smart autonomous decisions or otherwise interact with third parties independently;
International aspects
32. Notes the need also to consider amendments to international agreements such as the Vienna Convention on Road Traffic and the Hague Traffic Accident Convention;
33. Strongly encourages international cooperation in setting regulatory standards under the auspices of the United Nations;
34. Points out that the restrictions and conditions laid down in the 'Dual use regulation on the trade in dual-use items – goods, software and technology that can be used for both civilian and military applications and/or can contribute to the proliferation of weapons of mass destruction – should apply to applications of robotics as well;
Final aspects
35. Requests the Commission to submit, on the basis of Article 225 of the Treaty on the Functioning of the European Union, a proposal for a directive on civil law rules on robotics, following the detailed recommendations set out in the annex hereto;
36. Confirms that the recommendati ons respect fundamental rights and the principle of subsidiarity;
37. Considers that the requested proposal will not have any financial implications;
38. Instructs its President to forward this resolution and the accompanying detailed recommendations to the Commission and the Council.
