As an experienced kayak fisherman, guide, and sales shop, this is the number one question I get.  The second is "How do you keep from flipping?"

Well, these are related questions.  Let me walk you through the considerations for getting into the sport.

1)  The "boat" (kayak). 

Like anything else, you get what you pay for.  The more you pay for a fishing-specific kayak, the more durable, comfortable, and feature-rich it will be.  A $200 big-box kayak will get you on the water.  And if that is your budget, go for it!  But if you are thinking about getting serious, save the money.

Some kayak shops (such as Small Craft Outfitters) will offer free demos and/or offer rentals.  This is a great way to test a kayak before you spend the money, to make sure you enjoy it.

2)  Buy local!

There are many big-box and online retailers that offer kayaks cheap/to-your-door.  I can tell you from experience you get what you pay for.  Also, shipping a single kayak is an absolute nightmare.   A local shop has expertise in the kayaks they sell and deals with the nightmare that is shipping plastic boats.  A reputable retailer will work with you to ensure you get in the boat that fits your budget and needs.

3)  Paddles, kinda a big deal.

Like the boats, you get what you pay for with paddles.  Typically, the more you pay, the lighter paddle.  I personally love the Bending Branches line of paddles.  They are made here in Wisconsin (I have been to the factory several times and can confirm they do build them on-site).  They have excellent customer service, a great price-point, and are known internationally for making top-notch paddles.  

Is a $300 carbon fiber paddle worth the money?  Absolutely!  It makes a world of difference.  That said, not everyone is a hard-core kayak fishing guide or tournament angler.  Bending Branches has a variety available for all budgets.  At Small Craft Outfitters our number-one seller is the Bending Branches Angler Classic.  At $139.95, it is a great balance of value and weight.  Our second-best seller is the Angler Ace.  With a carbon shaft, it is the best value for durability and efficiency.

4) Accessories.  Load me up!.......?

I get asked about "critical" rigging options I recommend.  To new anglers and customers alike, I recommend keeping it basic.  Get a kayak, PFD, and paddle to fit your budget.  Then get a milk crate (available pretty much anywhere) and hit the water.  I have spent many-a-hours and many-a-dollars rigging my kayaks.  My first one was purchased in the winter.  I spent the winter and spring "making it perfect".  I got it on the water and completely changed everything.  Save yourself some time, money, and headaches.  Start simple and build from there.

5)  Safety........fifth?  

Absolutely not.  I am a volunteer EMT and personally rescued a 15 year old girl off a seemingly harmless river.  Safety is important.  There are several safety items to consider:

-Life jacket/PFD:  In Wisconsin, you aren't required to wear it.  But you must have one on the kayak somewhere.  You will NEVER see me, my family, my guides, my prostaff, or my clients without one on the water.  That said, it is a personal choice to make.

I personally love the Stohlquist line of life jackets.  They make a variety of paddle-specific kayaks for infants, children, recreational, whitewater, fishing, women-specific, and anything else you can imagine.  They are made to be smaller and cooler for comfort on the water while paddling (without compromising safety).

Some prefer the manual/auto inflating life jackets for their smaller size.  I personally do not want to worry about a manual or mechanical device in an emergency.  Again, to each their own.  But you will never see this style life jacket on my family, SCO team members, my clients, or I.  

-Other Safety Gear??

The life jacket is the most important safety device on a kayak (or any watercraft).  Below are some others to consider:

--Whistle/Air Horn:  To alert other kayakers/boaters to your location.  I have used mine several times.

--Visibility Flags/Lights:  I use a fiberglass rod with Green Bay Packers and Wisconsin Badger flags for visibility.  I also have a bright colored paddle to help signal others.  For lighting, I keep it simple with a single white light visible from 360 degrees.  I STRONGLY advise against the red/green navigation lights unless you are legally required and/or using a motor.  You don't want other powered watercraft thinking you are motorized, and thus, more maneuverable.

--Straight-blade knife:  I have a straight blade knife within reach at all times.  I used said knife to cut the girl I rescued on the river free.  Keep it handy (on your kayak rail or PFD) for easy access.  Stohlquist, and other brands, make a paddle sport specific knife with a dull tip for safety.  A straight blade is preferred over one that opens.  In an emergency, you don't want to rely on your brain or a mechanical process to deploy your blade.

--Throw bag:  A throw bag is an invaluable tool in your safety arsenal.  I have used mine once for a life-threatening rescue.  I have used it multiple other times in questionable situations.  Even if you don't buy a throw bag, bring some rope to toss to someone that may be in need.

--Survival gear:  Other things I carry are a first aid kit in a waterproof bag, a solar-powered floating light, a solar panel with USB ports for charging my devices, a CPR mask, a firestarter, toilet paper (probably my most used survival equipment), a LifeStraw (or iodine for water sanitation), and a space blanket.  It may be overkill for some.  But all of this is available for under $150 and may just save your life (or someone elses).

6)  Kayak Fishing X:  With the major growth of kayak fishing you'll find A LOT of products that are called Kayak Fishing X, where X is any product imaginable (rods, reels, lures, scents, etc.).  Other than the items mentioned above, don't buy into it right away.  I have used several of these products.  Some of them are great, others overpriced hype.  Ask around, try them out, and then decide if it is right fore you.

7)  Transport:  This is a big one.  The appeal of kayak fishing (other than the peace/lower cost/shallow water options) is the ease of transportation.  This said, fishing-specific kayaks can be wide and heavy.  Add accessories and they can be quite complex.  I have car-topped a Feelfree Lure 13.5 with ease.  But everyone has different budgets or physical abilities for transportation considerations.

I personally started with car-topping on a Yakima bike rack. I put foam blocks on the cross bars and used a bathroom mat on the trunk to slide the kayak up.  The bath mat doesn't slide on the vehicle while allowing the boat to slide with ease.  The foam on the cross bars keeps the boat from sliding around during transport and protects the hull from additional pressure of hard bars.

Another option is a bed extender for those who have trucks.  Bed extenders can be found inexpensively on Amazon and Harbor Freight.  It offers additional bed length/hull support (just be sure to adhere to red flag/light laws local to you since the kayak will be hanging beyond the factory bed).

Though I prefer a bed extender for solo trips, I often use a trailer for transport.  I have converted an old boat trailer with some redneck engineering (unistrut, PVC, and pool noodles) to accommodate 1-3 kayaks.  I also have an 8 x 12 ATV trailer with unistrut racks, a DIY rod tube made of 6" PVC and a Reese Towpower Conduit Kit, and an old truck toolbox to accommodate 1 - 9 kayaks.

Like everything else in this sport, it comes down to need, and ultimately, your budget.  

I welcome others with experience to post their beginners tips in this thread.  If you have any questions about getting into the sport, please post them below.  I am happy to help you out and save you time and money I have already spent trying to get up-and-running!