软件工程

软件工程复杂性

http://mperry.github.io/2012/07/26/software-engineering-is-hard.html

Accidental complexity can be addressed and resolved, e.g. the details of progamming in a lanuage. However, essential complexity is caused by the specification, design and testing of the software abstraction. The construction of the program and verification of the program to it’s specification is accidental complexity. Getting the abstraction right will always be hard with no silver bullets.

The components of essential complexity are:

• Complexity
• Conformity
• Changeability
• Invisibility

Complexity

Software is more complex than any other entity constructed by humans “because no two parts are alike”.

“Good software practice mitigates against duplication of parts since two parts that are similar are collapsed into a single callable routine in which the differences have been parameterized. In other more physical, human-built entities (bridges, buildings, cars, computers, etc.) the most obvious characteristic is the repetition of parts.” – Brooks

Advances in mathematics are due to ignoring complexity not relevant to the problem at hand. In software, ignoring complexities yields an incomplete program that does not solve the original problem. Yes, abstraction is essential for the decomposition of software. However, ultimately these ignored details must be considered. The complexity of software cannot be escaped.

Conformity

Whenever a software system is built, if anything needs to be bent to get the hardware, software, firmware, and peopleware to co-exist the software is that which changes. Software is far more malleable than are hardware, firmware, and people. Hardware and firmware come with predetermined behavior, and the behavior of human users is limited. Ultimately, the software is adapted to conform to the rest of the system.

Changeability

Software is subject to change far more than other technology. Once delivered, computer hardware and buildings change infrequently. The cost of changes is too large to do so. In practice, the cost of a change in software is high. The cost of maintaining a program over its lifetime is more than the cost of developing it and due to ripple effects and coupling, the cost of making a minor change can be major. However, a common perception is that the cost to make small changes is small.

Invisibility

Brooks observes that “software is invisible and unvisualisable”. For objects with a geometric reality, such as buildings, diagrams are faithful, complete, and useful representations from which inconsistencies and omissions can be gleaned. Software has no physical reality. Apart from the source code itself, software has no complete representation. Diagrams only capture particular aspects of a system. No set of diagrams captures all aspects of the system, their mutual inconsistency almost assured. Our ability to detect inconsistencies and omissions in them is limited.

软件工程的困难性

http://blog.prof.so/2012/05/top-10-software-engineering-challenges.html

I believe in the sustained relevance of ‘big agenda’ in software engineering: providing scientifically well-founded methods and tools to underpin the development of software systems that meet the needs of users, rapidly and at reduced cost. Better, faster, cheaper, in other words. Substantial technical progress has been made but, it could be argued, we are seeing some evidence of receding impact in those areas where that progress has significantly outstripped the capacity of practice to absorb innovation. This seems like an important prompt to review the key challenges we should be addressing and to ensure that they reflect the most outstanding immediate problems, hence this, entirely personal, top 10.

1. Relating Requirements and Architectures. It is clear that architectures cannot be directly derived (by refinement or other means) from a requirements specification. On the other hand the identification and elaboration of a suitable architecture are not independent of the requirements it must serve. The relation runs both ways with the architecture acting as a framework for the elicitation of requirements and shaping the space of possibilities that are afforded by the system. This complex, intertwined, co-development of architecture and requirements lies at the very core of software development and though we observe it, we do not understand it. 

2. Moving to ‘Evidence-based’ Practice. For the most part our existing state-of-practice is based on anecdote. It is, at its very best quasi-evidence-based. Few key decisions from the choice of an architecture to the configuration of tools and processes are based on a solid evidential foundation. To be truthful, software engineering is not taught by reference to evidence either. This is unacceptable in a discipline that aspires to engineering science. We must reconstruct software engineering around an evidence-based practice. 

3. Engineering Scalability. Increasingly we are required to build ‘Internet-scale’ services that must handle very large and rapid variations in the demand for resources. Existing practice is poor. We have some high level patterns for limited classes of application. Mostly, the best answers we can offer are resource profligacy, throwing hardware at the problem, or 'suck it and see', basically putting up a system and systematically fixing it each time it falls over. It is unclear to me whether our existing mathematical and analytic tools are able to give us purchase on scalability. 

4. Addressing Semantic Divergence. We operate in a context where standards for modelling and information exchange are very important. Unfortunately software engineering standards, principally promulgated by the OMG, and web standards, principally promulgated by the W3C, have been moving in different incompatible directions. The idea that the (semantic) representations at 'run-time' should be different to the (semantic) representations at 'development time' seems to me wrong, wasteful of effort and profligate of technical opportunity. We need to address this. 

5. Confident Estimation. We are very poor at answering the first question that any client asks: 'how much is this going to cost me?' We  cannot reliably predict the cost/effort required to build a system. We may be fortunate and have built a very similar system before in which case we can make a rough guess, otherwise we are clueless. Function Points, the state of the art, are precious little assistance. Other estimation schemes only work for small systems, relatively ‘late’ in the process. Software economics needs a radical rethink. We need to integrate cost into software engineering in a fundamentally new way. 

6. Engineering the Cloud. We know how to build Software-as-a-Service (SaaS) applications. Sort of, see my reservations about scalability. We don’t however know how to: buy it, manage QoS (Quality of Service) or achieve interoperability among SaaS offerings. SaaS is not simply 'yet-another-software-architecture' it is bound together with a set of new business models that require an engineering response. 

7. Building ‘Apps’. Apps have fundamentally changed how services are likely to be consumed. Fragmentary, highly-tuned, device-specific interfaces that bridge across to SaaS with managed data ‘sync’ to keep context, are becoming a very important development target. Not least because a viable payment model exists. We have only scratched the surface of the engineering challenges: app stores, app management, app assembly or composition, all require attention.  

8. Developing ‘Adaptive’ Systems. We have problems building systems that must exhibit robustness to a changing environment; problems building systems embedding significant COTS/Community Sourced independently evolving components; problems building systems that involve user scripting and ‘plug-ability’. In short, problems building the sort of systems we are called on to construct all the time. We need to develop engineering models and methods for assembling software systems that can dynamically adapt to context and can ‘account for themselves’. 

9. Reconstructing Governance. I have been struck by the tendency in software engineering for us to repeat our mistakes. More specifically to repeat mistakes that we understand are mistakes and which we possess the knowledge to avoid. We are driven to this, I contend, by misalignments at the boundaries between business and software engineering that is at the point where business value and the requirements of an engineering process collide. This is the domain of governance, which is poorly understood and little studied. 

10. Rethinking Software Production. Software development is no longer garage ‘design and make’. Most software products and services are embedded in a network of complex inter-product and inter-supplier dependencies. Software is the result of the operation of a 'supply chain' that must be designed and forms part of an 'ecosystem' that must be accommodated. Rethinking software production requires a new discipline of business model and software system co-design.