truthsharer – While I do think the effect of sherry casks is often overstated, to say that, after cleaning, etc., it is no different than a beer barrel is simply incorrect. For one, sherry casks are made from Quercus Robur wood (aka Spanish Oak). This type of wood imparts a much different flavor than Quercus Alba (White American oak). Next, as a general rule, sherried whiskies are sweeter than those aged in purely ex-bourbon casks. Try Glenlivet 12 (primarily ex-bourbon), and then try Macallan 12 (100% sherry); the difference in level of sweetness is largely due to the sherry casks.
Finally, many single cask bottlings are denoted as being stored in “first-fill” sherry casks (the first time it was filled after it stored sherry), while others are denoted as being stored in “re-fill” sherry casks (held sherry and then whisky before being refilled again with whisky). According to what you are saying, there would be no point in doing this. Of course, there is, and the reason is because the sherry cask (aside from the Robur vs. Alba wood differences) influences the whisky to a greater degree the first time it is filled vs. subsequent times.