Best practices for installing and configuring apps in a distributed Splunk install?

Here's how I do split up an "app". Basically, I break it up into multiple Splunk apps, so that the right pieces can be sent to the right place. For most apps, I'd have:

• myapp_inputs: This would contain the input configurations and is sent to forwarders. Mostly it's in inputs.conf, but would include scripted inputs and the input-time props.conf settings. (See this for what is input-time configuration.)
• myapp_props: This basically contains index-time and search-time data properties. I usually send it to all servers (forwarders, indexers, search heads) because it doesn't hurt to have the extra configurations, and it saves having to worry about whether a particular configuration is search-time or input-time or index-time. It also lets you forget about whether a forwarder is light or heavy. Basically, this is stuff in props.conf and things referenced from there. (e.g., transforms.conf, lookup tables and scripts, search scripts)
• myapp_summarize: Goes on the search head that runs summary jobs. This has the summary searches that generate summary index data.
• myapp: This contains dashboards and views, and just goes to the search head. It will also include most saved searches, and usually includes alerts. Sometimes i'll put the views in the myapp_props and myapp_summarize apps instead. I'll include some of the search-time configurations here, if those configurations are more purely view-oriented rather than data-oriented (e.g., lookup tables used to populate form dropdowns, vs lookup tables used to enrich events).

I suppose there could be app-specific configs for something like data routing, but I haven't seen that configuration at application-level (vs being managed at the base server server level). Alerts and (non-summary) scheduled searches may be split from the myapp app into a myapp_jobs app, but that's only of benefit if you're getting to the point where you have scaled to a separate search head for users vs jobs.

This is recommended (by me) even if you have a non-distributed Splunk install, as it:

• gives the flexibility to add and reconfigure your system, e.g., add a search head, split summarization away, change forwarders from heavy to light, without having to change and split up config files
• corresponds roughly to different deployment phases, and different tasks/roles/people who will be involved at each phase; e.g., dashboards and views are usually handled in a different role or by different people from the data inputs.

Lately, I have been also splitting up the props into two apps, myapp_props and myapp_indexprops to distinguish search-time from index/parse-time configurations. I am not as certain that this is that important to separate however.