Brand marketing and new product development have radically changed from the days portrayed by Don Draper and his team in AMC's "Mad Men" TV series. Back in the '50s and '60s, mass marketing/advertising/new products seemed to be more art than science. Intuition, gut feel, and luck seemed to have more to do with the success of brands and new products than did a disciplined process.

However, since then, sophisticated analytical and process models have come to the fore. Though many of the new tools and approaches have their place, I would argue that some have come at the expense of solid strategic and creative thinking.

That gap in thinking means that brand and new product innovations have not been as "disruptive" as they were in the past—and they lack real competitive advantages. Among the reasons for that are the following:

• Retailer and manufacturer consolidation. Unfortunately, stock price pressures from Wall Street have created a myopic view of how to effectively manage and grow businesses—particularly in consumer product companies. The number of retailers and manufacturers serving consumers has radically dwindled since the '60s. A lot of companies and retailers either no longer exist or are now part of bigger concerns. The result has been lower headcount, lower tolerance for long-term payback, and lower new product activity.
• The retail landscape continues to shift. Brick and mortar outlets have been losing share to online outlets because of consumer "showrooming" purchase behaviors. Moreover, private label/control brands now comprise roughly 25% of all retail sales dollars, according to the Private Label Manufacturers Association, thus decreasing the importance of national brands. Finally, lack of store buyer continuity and buying habits continue to put downward pressure on retail prices—even if a disruptive new product is launched.
• Consumer demographics have shifted. Millennials are changing the face of marketing forever, according to the Boston Consulting Group. Millennials will likely outnumber Baby Boomers 78 million to 56 million by 2030. This ascendant segment will have habits and purchase behaviors different from those of Boomers; in turn, that will have an impact on the marketing activities intended to reach them.
• Marketing and Information Technology are merging. The growth of social media, along with more sophisticated analytical tools, is bringing these two functions together. Since more media is now both traditional and digital, older marketers need to better understand social media and tactics and become more "tech savvy," while younger marketers need to gain a better understanding of strategy. Allocation of marketing dollars to the most effective media channel for driving sales is critical.
• Idea generation/intellectual property have become too narrowly defined. Traditionally, idea generation and intellectual property were sequential activities—you had to have good ideas before you had a patentable intellectual property. However, some companies have recognized good ideas can come from anywhere—both from inside and outside. As a result, they have formalized their idea generation and intellectual properties processes to ensure they have both sustainable and robust streams of new product/business ideas (such as P&G).

As a result of those trends, many consumer product companies are losing or have already lost their competitive edge and advantages, and they are not really innovating any longer. The evidence is in Boston Consulting Group's whitepaper titled "The Most Innovative Companies 2013."

In this report, BCG continues to track the state of innovation via interviews with 1,500 senior executives who rate and rank their own companies as well as the competition. The report ranks the 50 most innovative companies and reports their movement up/down this list over time. The sad fact from the latest report is that only five consumer product companies are listed among the top 50: