Why do planets move in an elliptical orbit?

Not sure if you're looking for a more mathematical answer or just the "why", but to answer the why, I'll start with some history on this.

Everyone who worked out a model for the Solar System, from Aristotle to Copernicus, liked circles. Even though Copernicus correctly reasoned that the Earth moved around the Sun and not the Sun around the Earth, he continued to use circles in his models of the motion of the planets.

After Copernicus, Tycho Brahe, funded by the King of Denmark, had the best equipment at the time for observing the motion of the stars and planets and he was able to make star charts that were ten times as accurate as anyone before him. Brahe used equipment like this mural quadrant, and a large private observatory to take extremely accurate records.

Kepler, who was a better mathematician than Brahe, desperately wanted to get his hands on Brahe's star charts and the use of his observatory and equipment (so much so that when Brahe died, there were rumors that Kepler had poisoned him, though that probably didn't happen). When Kepler finally had everything at his disposal, he was able to work things out and study the Solar System more accurately. However, he still didn't know why the planets moved in ellipses; he'd only worked out that the ellipses fit the movement so well that it almost certainly had to be true, but he had no idea why.

Kepler, in fact, didn't care for ellipses. He liked circles better, but he couldn't deny that ellipses worked. Source.

Nobody knew why planets moved in ellipses until Isaac Newton was asked that question and had to invent calculus to answer it. Calculus explains why planets orbit in ellipses, and that's the real answer.

If "calculus" isn't a satisfying answer, a way to kind of explain it would be to throw a penny out of a space shuttle (which isn't a good idea, but let's say you do). As the penny falls towards the Earth, it falls faster and faster (if we ignore air resistance) until it hits the ground.

Now, if you fling the penny from the space shuttle at a much faster speed and at a different angle, so that it only gets near Earth and misses the planet, it would actually start to orbit Earth. It would fall faster and faster until it passes the Earth, and then, like shooting a bullet into the air, the penny will slow down as it flies past the Earth.

According to Kepler's second law, the penny's fastest speed is at the point closest to the Earth (the perigee). That's essentially how objects in orbits work: as they move closer to the body they orbit, they accelerate faster and faster. Our penny will get so fast that, once it comes around the planet, it will be flung very far away, which will then slow it down. This is what creates an elliptical orbit.

Its motion is like a spring, falling toward the planet then flying away, but at the same time, orbiting in a circular motion with the spring motion, with 1 period per orbit. That motion of moving closer and then further in each orbit forms an ellipse.

It makes the most sense if you think of the velocity being greatest at the closest point and lowest at the furthest point. The low velocity moves it closer while the high velocity moves it back out further. The total energy of the object in orbit (kinetic energy plus potential energy) remains constant.