Research Philosophy

Some machine learning philosophy

The bitter lesson by Rich Sutton

Seeking an improvement that makes a difference in the shorter term, researchers seek to leverage their human knowledge of the domain, but the only thing that matters in the long run is the leveraging of computation.

ICLR2019 Keynote by Leon Bottou

  • The statistical problem is only a proxy.
  • Nature does not shuffle the examples. We shouldn’t.
  • Invariance across environments buys extrapolation powers.
  • Invariance across environments is related to causation.
  • Invariant representations enable invariance.
  • Realizable problems bring different challenges.

 

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s