Executive Oath of Office: “I do solemnly swear (or affirm) that I will faithfully execute the Office of President of the United States, and will to the best of my ability, preserve, protect and defend the Constitution of the United States.” 
—United States Constitution, Article II, Section 1, Clause 8 

It is traditional for the President to give a speech after he is sworn in on Inauguration Day.  This is called an Inaugural Address.  A Vice-President who becomes President upon the death or resignation of the President does not give an Inaugural Address.  Hence, several American Presidents (such as John Tyler and Gerald Ford) are not on this list.  Policy goals of the new President are often presented in the Inaugural Address.  References to recent events are also noted.  Both George W. Bush and Rutherford B. Hayes made references to the disputed elections that brought both to power during an Inaugural Address.

Full-text of Inaugural Addresses:

George Washington 1789, 1793 

John Adams 1797 

Thomas Jefferson 1801, 1805 

James Madison 1809, 1813

James Monroe 1817, 1821

John Quincy Adams 1825

Andrew Jackson 1829, 1833

Martin Van Buren 1837 

William Henry Harrison 1841

James Polk 1845 

Zachary Taylor 1849

Franklin Pierce 1853

James Buchanan 1857

Abraham Lincoln 1861, 1865 

Ulysses S. Grant 1869, 1873

Rutherford B. Hayes 1877

James A. Garfield 1881

Grover Cleveland 1885, 1893

Benjamin Harrison 1889

William McKinley 1897, 1901

Theodore Roosevelt 1905

William Howard Taft 1909 

Woodrow Wilson 1913, 1917

Warren G. Harding 1921 

Calvin Coolidge 1925 

Herbert Hoover 1929

Franklin D. Roosevelt 1933, 1937, 1941, 1945

Harry S. Truman 1949

Dwight D. Eisenhower 1953, 1957

John F. Kennedy 1961

Lyndon Baines Johnson 1965

Richard Milhous Nixon 1969, 1973

Jimmy Carter 1977 

Ronald Reagan 1981, 1985 

George Bush 1989 

William Clinton 1993, 1997

George W. Bush 2001, 2005