https://jiayuanm.com/projects/universal-agent/

1<!DOCTYPE html>
2<html lang="en">
3<head>
4    <title>Universal Agent</title>
5    <meta charset="utf-8">
6    <meta http-equiv="X-UA-Compatible" content="IE=edge">
7    <meta name="viewport" content="width=device-width, initial-scale=1, shrink-to-fit=no">
8    <link rel="stylesheet" href="https://maxcdn.bootstrapcdn.com/bootstrap/4.0.0-alpha.6/css/bootstrap.min.css" integrity="sha384-rwoIResjU2yc3z8GV/NPeZWAv56rSmLldC3R/AZzGRnGxQQKnKkoFVhFQhNUwEyJ" crossorigin="anonymous">
9    <link rel="stylesheet" href="https://maxcdn.bootstrapcdn.com/font-awesome/4.4.0/css/font-awesome.min.css">
10
11    <!--[if lt IE 9]>
12      

12<script src="http://cdn.bootcss.com/html5shiv/3.7.2/html5shiv.min.js"></script>

12
13      

13<script src="http://cdn.bootcss.com/respond.js/1.4.2/respond.min.js"></script>

13
14    <![endif]-->
15    <style type="text/css">
16body{margin:0;padding:0;background-color:#eee}
17#header{background:#ddd;opacity:0.95;margin:0 auto;padding:50px 0 0;text-align:center;cursor:default;text-align:center}
18#header-container{margin:0 auto;padding: 0 2em;max-width:1080px}
19
20#header-container .col-logo{text-align:left}
21#header-container .logo{position:relative;z-index:100;height:50px;margin-top:20px;margin-right:10px}
22
23#header-container .col-info{text-align:left}
24#header-container .textbox-info{display:inline-block;margin:0 auto;text-align:left}
25#header-container .textbox-info #title{margin:.525em 0;color:#005ca1;font-size:1.60em;font-family:Source Sans Pro,Helvetica,sans-serif} #link-ul{margin:30px auto auto;padding:0;list-style:none;text-align:center} .link-li{display:inline-block;margin:auto;padding:0 10px;color:#ccc;line-height:10px}
26#header-container .textbox-info p{line-height:12px}
27#header-container .textbox-info .email{color:#666; font-size:14px;}
28#wave-canvas{display:block;margin:-80px 0 0;width:100%;height:150px}
29
30#content{padding-top:0px;text-align:left}
31#content-container{margin:0 auto;padding: 0 2em;max-width:1080px;color:#333333}
32
33#content-container .header{background:#f4f4f4;padding:15px 30px 5px;border-bottom:3px solid #dddddd}
34#content-container .header .indicator{color:#777777;margin-right:12px}
35
36#content-container .content{background:#ffffff;padding:15px 5px 15px 30px}
37#content-container .content .caption{color:#666666}
38#content-container .content .bib{color:#666666;font-size:14px;}
39#content-container .content .bib pre{margin:0;padding:0;}
40
41#footer{padding:2em 0 0.5em;margin:30px 0 0;background:#f4f4f4;opacity:0.95;font-size:14px;line-height:12px;text-align:center}
42    </style>
43</head>
44<body>
45<div id="main">
46    <div id="header">
47        <div id="header-container" class="container">
48          <div class="row">
49            <div class="col-md-12 col-xl-4 col-logo">
50                <img src="data/img/logo-thu.png" class="logo" />
51                <img src="data/img/logo-USC-full.png" class="logo" />
52            </div>
53            <div class="col-md-12 col-xl-8 col-info">
54                <div class="textbox-info">
55                    <h1 id="title">Universal Agent for Disentangling Environments and Tasks</h1>
56                    <p>
57                    <a href="http://jiayuanm.com">Jiayuan Mao</a>,
58                    <a href="https://github.com/dhh1995">Honghua Dong</a>,
59                    <a href="http://www-bcf.usc.edu/~limjj/">Joseph J. Lim</a>
60                    </p>
61                    <p class="conference">In <a href="http://iclr.cc">International Conference on Learning Representations</a> (2018)</p>
62                    <p class="links">Links:
63                    <a href="https://openreview.net/forum?id=B1mvVm-C-">Paper</a>
64                    <!--[arXiv]-->
65                    <!--[Code]-->
66                    </p>
67                    <p class="email">maojiayuan [at] gmail.com, dhh19951 [at] gmail.com, limjj [at] usc.edu</p>
68                </div>
69            </div>
70          </div>
71
72        </div>
73        <canvas id="wave-canvas"></canvas>
74    </div>
75    <div id="content">
76        <div id="content-container" class="container">
77            <div class="header"><h4><span class="indicator">=</span>Overview</h4></div>
78            <div class="content">
79            <p>
80            Recent state-of-the-art reinforcement learning algorithms are trained under the goal of excelling in one specific task.
81            Hence, both environment and task specific knowledge are entangled into one framework.
82            However, there are often scenarios where the environment (e.g. the physical world) is fixed while only the target task changes.
83            Hence, borrowing the idea from hierarchical reinforcement learning, we propose a framework that disentangles task and environment specific knowledge by separating them into two units.
84            The environment-specific unit handles how to move from one state to the target state; and the task-specific unit plans for the next target state given a specific task.
85            The extensive results in simulators indicate that our method can efficie

85ntly separate and learn two independent units, and also adapt to a new task more efficiently than the state-of-the-art methods.
86            </p>
87            <p>
88            <b>TL;DR</b>: We propose a DRL framework that disentangles task and environment specific knowledge.
89            </p>
90            </div>
91        </div>
92        <div id="content-container" class="container">
93            <div class="header"><h4><span class="indicator">=</span>Framework</h4></div>
94            <div class="content">
95            <p>
96            <center>
97            <img width="80%" src="data/img/ua.png"/>
98            </center>
99            </p>
100            <p class="caption">
101            Figure1: Proposed <i>Universal Agent</i>, which consists of three parts: a perception function (phi) mapping raw observation to feature space,
102            a path function as an environment actor, and a goal function (tau) for future state planning.
103            </p>
104
105            <dl class="row">
106                <dt class="col-sm-3">The perception module (<i>phi</i>)</dt>
107                <dd class="col-sm-9">Given the raw observation of the state, the perception function <i>phi</i> encodes the observation into a feature space.
108                It can be jointly optimized with <i>path</i>, or separately obtained (e.g., Auto-Encoder).</dd>
109                <dt class="col-sm-3">The environment-specific module (<i>path</i>)</dt>
110                <dd class="col-sm-9">
111                Given a <i>(current state s, goal state s')</i> pair. The <i>path</i> function outputs a probability distribution over the action space for the first action to take
112                at state s in order to reach state s'.
113                </dd>
114                <dt class="col-sm-3">The task-specific module (<i>tau</i>)</dt>
115                <dd class="col-sm-9">
116                Given the current state. the goal function (<i>tau</i>) determines what the goal state should be for a specific task.
117                The <i>path</i> function is then invoked to get the next primitive action.
118                </dd>
119            </dl>
120            </div>
121        </div>
122        <div id="content-container" class="container">
123            <div class="header"><h4><span class="indicator">=</span>Resources</h4></div>
124            <div class="content">
125            If you find this project useful, please consider citing:
126            <blockquote class="blockquote bib">
127            <pre>@inproceedings{
128    mao2018universal,
129    title={Universal Agent for Disentangling Environments and Tasks},
130    author={Jiayuan Mao and Honghua Dong and Joseph J. Lim},
131    booktitle={International Conference on Learning Representations},
132    year={2018},
133    url={https://openreview.net/forum?id=B1mvVm-C-},
134}</pre>
135            </blockquote>
136            </div>
137        </div>
138    </div>
139    <div id="footer">
140        <p>The Paper Authors © 2018</p>
141    </div>
142</div>
143
144<!-- jQuery first, then Tether, then Bootstrap JS. -->

145<script src="https://code.jquery.com/jquery-3.1.1.min.js" crossorigin="anonymous"></script>

145

146<script src="https://cdnjs.cloudflare.com/ajax/libs/tether/1.4.0/js/tether.min.js"
147        integrity="sha384-DztdAPBWPRXSA/3eYEEUWrWCy7G5KFbe8fFjk5JAIxUYHKkDx6Qin1DkWx51bBrb" crossorigin="anonymous"></script>

147

148<script src="https://maxcdn.bootstrapcdn.com/bootstrap/4.0.0-alpha.6/js/bootstrap.min.js"
149        integrity="sha384-vBWWzlZJ8ea9aCX4pEW3rVHjgjt7zpkNpZk+02D9phzyeVkE+jo0ieGizqPLForn" crossorigin="anonymous"></script>

149

150<script type="text/javascript" src="static/js/jquery.color.min.js"></script>

150

151<script type="text/javascript" src="static/js/wave.js"></script>

151

152<script type="text/javascript">
153$(function() {
154    targetColor = $("#title").css("color")
155    animatedLink = function(speed) {
156        $(".link-li").hover(function() {
157            $(this).find('.icon').animate({
158                color: targetColor,
159                borderColor: targetColor
160            }, speed);
161            $(this).find('.caption').animate({
162                color: '#798350'
163            })
164        }, function() {
165            $(this).find('.icon').animate({
166                borderColor: '#cccccc',
167                color: '#cccccc'
168            }, speed);
169            $(this).find('.caption').animate({
170                color: '#cccccc'
171            })
172        })
173    };
174    // fullBg();
175    animatedLink(400)
176});
177</script>

177
178</body>
179</html>