本报告所述工作，是作为 康奈尔航空实验室有限公司内部研究计划 的一部分开展的。本文所讨论的概念，源自作者在 生理心理学（physiological psychology） 领域开展的一些独立研究。这些研究旨在建立一种可用于分析的 脑类比模型（brain analogue） 。作者对这一研究领域的持续关注已有五六年之久。 感知机（perceptron）概念是这一研究计划近期形成的成果；目前所开展的工作旨在确定感知机在技术和经济方面的可行性。

The work described in this report was supported as a part of the internal research program of the Cornell Aeronautical Laboratory, Inc. The concepts discussed had their origins in some independent research by the author in the field of physiological psychology, in which the aim has been to formulate a brain analogue useful in analysis. This area of research has been of active interest to the author for five or six years. The perceptron concept is a recent product of this research program; the current effort is aimed at establishing the technical and economic feasibility of the perceptron.

自电子计算机和现代伺服系统问世以来，人们越来越关注这样一个问题： 能否构造出一种具有类似人类功能的装置，使其具备感知、识别、概念形成以及从经验中进行概括的能力。 尤其受到关注的是这样一种机器：它能够直接对来自物理环境的光、声音、温度等信息进行概念化处理——即我们每个人都熟悉的 “现象世界” （phenomenal world）——而不需要人工介入，由人来对必要的信息进行整理和编码。

这类系统首先必须具备的一项基本能力，是 能够识别在现象上相似，或在经验上具有联系的复杂信息模式 ；这一过程对应于心理学中的 “联想” （association）和 “刺激泛化” （stimulus generalization）现象。系统必须能够识别 “同一个” 物体，即使该物体处于不同的方向、具有不同的尺寸和颜色，或经历了不同的变换，并且即使其处于各种不同的背景之中，也仍然能够完成识别。

对于 “相似” 形式的识别，在一定程度上可以通过数字计算机或模拟计算机上的分析程序来实现。然而，很难设想出一种具有普遍适用性的分析程序，例如能够从任意角度、以任意姿态或处于任意位置时，都识别出一个人的形态，而不必实际存储一个庞大的 参考图形库 ，再将感知到的对象与其中的图形进行比较。一般而言，这类身份或形式的识别必须通过 学习 获得，即从经验中习得；而如果系统还必须具备较高的经济性，那么 存储系统或记忆系统 中的功能单元数量，就应当远远少于需要保存的形式或记忆的数量。

正是最后这一要求，似乎与传统计算机系统的性质存在不相容之处。此外，如果一个存储了数十万乃至数百万种模式的记忆系统，必须通过 顺序扫描 的方式逐一检索，才能识别某个对象，那么传统系统所需要的处理时间将变得极为庞大。因此，所提出的系统不仅必须节省存储空间；它还必须能够“直接”对一个对象进行分类或识别——也就是说，它必须能够在不采用顺序搜索程序的情况下，直接定位与该对象相关的记忆信息。

Since the advent of electronic computers and modern servosystems, an increasing amount of attention has been focused on the feasibility of constructing a device possessing such human-like functions as perception, recognition, concept formation, and the ability to generalize from experience. In particular, interest has centered on the idea of a machine which would be capable of conceptualizing inputs impinging directly from the physical environment of light, sound, temperature, etc.——the "phenomenal world" with which we are all familiar——rather than requiring the intervention of a human agent to digest and code the necessary information.

A primary requirement of such a system is that it must be able to recognize complex patterns of information which are phenomenally similar, or are experientally related——a process which corresponds to the psychological phenomena of "association" and "stimulus generalization". The system must recognize the "same" object in different orientations, sizes, colors, or transformations, and against a variety of different backgrounds. The recognition of "similar" forms can be carried out, to a certain extent, by analytic procedures on a digital or analog computer, but it is hard to conceive of a general analytic program which would, for example, recognize the form of a man seen from any angle, and in any posture or position, without actually storing a large library of reference figures against which the percept could be compared. In general, identities of this sort must be learned , or acquired from experience, and if the system is to be economical, the number of functional units in the storage system, or memory, should be much less than the number of forms or memories to be retained. It is this last requirement which seems to be incompatible with the nature of conventional computer systems. Moreover, if a memory with hundreds of thousands or millions of patterns stored in it must be scanned sequentially in order to identify an object, the time required by conventional systems becomes excessive. The proposed system must not only economize on storage space: it must be able to categorize or identify an object "directly" ——i.e., it must locate the relevant memories without resorting to a sequential search procedure.

近期由作者本人开展的理论研究表明， 构造一种电子式或机电式系统是可行的 。这种系统能够学习识别光学、电气或音调信息模式之间的相似性或同一性，其工作方式可能与生物脑的感知过程具有密切的类比关系。所提出的系统在运行过程中采用的并非确定性原理，而是 概率性原理 ；其可靠性则来源于从大量单元构成的总体中获得的 统计测量结果的性质 。按照这些原理运行的系统，将被称为 感知机（perceptron） 。一种被设计用于接受光学或“视觉”模式作为输入的感知机，将被称为 光电感知机（photoperceptron） ；接受音调模式或“听觉”输入的，则称为 声感知机（phonoperceptron） 。此外，我们还可以考虑构造电感知机（electroperceptron）或无线电感知机（radioperceptron）的可能性，并为其配备相应的感觉装置。此外，还有必要对瞬时刺激感知机（momentary stimulus perceptron）和时间模式感知机（temporal pattern perceptron）加以区分。后者 具有记忆事件时间序列 的能力，而不是仅仅记忆瞬时出现的短暂图像；后者例如可以来自一段电影胶片中截取的一组彼此孤立的画面。

本文目前的讨论将集中于一种瞬时刺激光电感知机（momentary stimulus photoperceptron）。就这一类系统而言，它似乎是能够被实际构建、并用于展示其一般原理的最基本装置。我们建议构建这一模型，其目的不仅在于 验证感知机概念的可行性 ，同时也将其作为一种研究工具，用于进一步研究该系统所采用的各种原理。

Recent theoretical studies by this writer indicate that it should be feasible to construct an electronic or electromechanical system which will learn to recognize similarities or identities between patterns of optical, electrical, or tonal information, in a manner which may be closely analogous to the perceptual processes of a biological brain. The proposed system depends on probabilistic rather than deterministic principles for its operation, and gains its reliability from the properties of statistical measurements obtained from large populations of elements. A system which operates according to these principles will be called a perceptron . A model which is designed to accept optical, or "visual" patterns as inputs will becalled a photoperceptron . One which accepts tonal patterns, or "auditory" inputs, will be designated a phonoperceptron , and we might also consider the possibility of electro- or radioperceptrons, with corresponding sensory devices. It is also useful to distinguish between momentary stimulus perceptrons and temporal pattern perceptrons- the latter having the ability to remember temporal sequences of events, rather than transient momentary images, such as would be obtained from a collection of isolated frames cut from a strip of movie film.

The present discussion will concentrate on a momentary stimulus photoperceptron , which seems to be the most elementary device which could be built to demonstrate the general principles of this type of system. It is suggested that the proposed model should be built, not only to demonstrate the workability of the perceptron concept, but as a research tool for further study of the principles employed.

我们可以将感知机视为一个 黑箱 ：其输入端连接一台电视摄像机，输出端则连接一台字母打印机或一组信号灯。这样一来，它的工作过程就可以描述为：通过学习，使系统能够对所有属于某个任意定义类别的光学刺激，产生相同的输出信号（或打印出相同的文字）。

例如，这样一个类别可以是：在电视摄像机视野范围内，三角形所发生的所有二维平移；也可以是：某个三维形体所处的各种旋转位置。

对于一个刺激模式，如果其中各种形式需要被识别为 等价形式 ，那么每一种形式都可以表示为电视扫描栅格（TV raster）中一组独特的发光点。所有这些“等价”形式共同构成一个 变换集合 （transposition set），我们将其称为 T。

我们可以通过以下方式教会系统区分两种这样的广义形式，或称“感知对象（percepts）”：对于每一种形式，从其所有可能变换所构成的集合中随机抽取样本并呈现给系统，同时“强制”系统分别对两种形式作出指定的响应——对于形式1输出 Output 1，对于形式2输出 Output 2。

例如，我们可以要求感知机学习“正方形”和“圆形”这两个概念，并规定：当识别为“正方形”时点亮信号灯1，识别为“圆形”时点亮信号灯2。

接下来，我们可以向系统展示大量不同大小、位于不同位置的正方形，同时保持信号灯1亮起，从而“强制”系统产生相应的响应。随后，再向系统展示一组类似的圆形，同时保持信号灯2亮起。

如果此时再向感知机展示任意一个正方形或任意一个圆形，我们预期它都能够点亮相应的信号灯，并且具有较高的正确响应概率。

采用统计学原理设计这一系统所需要付出的代价，是系统在任何一个具体情况下都存在产生 错误响应的概率 ——也就是说，这种错误并非由于某个组成部件发生故障而造成，而是 系统本身的性质所固有的错误概率 。

不过，目前看来，这一概率可以降低到一个不高于 电子设备因部件故障而产生典型错误的概率 的水平。

We might consider the perceptron as a black box, with a TV camera for input, and an alphabetic printer or a set of signal lights as output. Its performance can then be described as a process of learning to give the same output signal (or print the same word)for all optical stimuli which belong to some arbitrarily constituted class. Such a class might be the set of all two-dimensional trans-positions of a triangle over the field of view of the TV camera, or the set of rotational positions of a 3-dimensional form. The forms of a stimulus pattern which are to be identified as equivalent can each be represented as a unique set of illuminated points in the TV raster. All of the "equivalent" forms constitute a transposition set , which we will call T.

It is possible to teach the system to discriminate two such generalized forms, or "percepts", by presenting for each form a random sample from the set of its possible transformations, while simultaneously "forcing" the system to respond with Output 1 for Form 1, and Output 2 for Form 2. For example, we might require the perceptron to learn the concepts "square" and "circle", and to turn on Signal Light 1 for "square", and Signal Light 2 for "circle". We would then proceed to show the system a large set of squares of different sizes, in different locations, while holding Light No. 1 on, thus "forcing" the response. We would then show a similar set of circles, while holding Light No. 2 on. If we then show the perceptron any square or any circle, we would expect it to turn on the appropriate light, with a high probability of being correct. The penalty that we pay for the use of statistical principles in the design of the system is a probability that we may get a wrong response in any particular case -- i.e., a wrong response that is inherent in the nature of the system, rather than due to a malfunction of one of its components. It appears that this probability may be reduced, however, to a quantity no greater than the typical probabilities of an error due to malfunctions in electronic equipment.

系统主要由三个部分组成，如图1所示：

（1）S系统

（感知系统，Sensory System）

S系统可以表示为电视扫描栅格中的一组点，或者表示为一组光电池。S系统中的每一个栅格点，都与A系统（关联系统，Association System）中的若干单元相连接。当某个S点处于“开启”状态（即受到光照）时，就会向这些单元传送脉冲。

任意一条具体的S—A连接，都可以是正连接（向A单元传递正向或“兴奋性”信号），也可以是负连接（倾向于抑制或压制A单元的活动）。例如，S栅格中的某一个点，可以与10个A单元建立正连接，同时与另外10个A单元建立负连接。

（2）A系统

（关联系统，Association System）

A系统负责执行 输入与输出之间的切换功能 。每一个A单元都会接收来自若干S点的脉冲，并将输出传送至一个或多个响应单元（R单元，Response Units）。

A单元具有两个特征参数：固定参数 θ：即阈值（threshold value），对应于引发输出所必需的输入脉冲代数和；随机变量 v：即“输出值（output value）”，可以是输出脉冲某种能够进行物理测量的特性，例如振幅、频率或延迟时间。A单元输出值的大小会随着其历史状态（history）而发生变化，并因此承担系统记忆功能中的计数器或寄存器作用。

（3）R系统

（响应系统，Response System）

R系统通常由数量相对较少的单元组成，这些单元可以驱动字模条（type-bars）或信号灯。当从A系统接收到的信号的平均值或净值超过临界值 θr 时，相应的R单元就会被激活。

除了打印或显示输出信号之外，每一个响应单元还会反馈脉冲，用以抑制或切断所有相互排斥的R单元，以及可能激活这些R单元的A单元的活动。这些抑制性反馈连接（inhibitory feedback connections）确保，在一组彼此互斥的响应单元中，任何时刻只能有一个响应被触发。

响应电路的设计方式是：如果脉冲同时到达两个R单元，那么输入信号平均值（或净值）较大的那个单元将首先作出响应；它随后通过自身的抑制性连接切断另一个单元，使后者在被触发之前即受到抑制。

因此，一整组彼此互斥的R单元，可以发挥类似于一个多稳态触发器（multi-stable flip-flop）的作用，在任何时刻只能有其中一个状态处于“开启”状态。这样的彼此互斥R单元组可以有若干组并行存在。例如，为了打印一个十字母单词，可以设置十组这样的单元，每一组负责一个字母位置。在这种情况下，系统的组织方式如图2所示，每一个R组都对应一个独立的A组。

在图3中，则进一步对系统进行了详细分解，展示了一种被设计用于输出一个三位二进制数的感知机，其逻辑组成结构。这个数中的每一个二进制位，都由一个包含两个成员的R组表示，即0和1。

在图3中，圆形表示功能单元的集合，矩形表示单个单元。该图展示了当响应“101”处于“开启”状态时，抑制性连接所产生的作用。只要这一响应处于活动状态，图中所有的阴影区域都会受到抑制。需要注意的是，一个处于活动状态的A组与一个受到抑制的A组之间的交集并不会受到抑制。换言之，抑制性连接是从每一个R单元出发，只连接到那些并未向该R单元提供连接的A子集。

这种系统的整体组织结构，可以通过一张表格，或者说通过一个表示各单元之间连接关系的连接矩阵（matrix of connections）来描述。例如，图3所示的系统，就可以用类似图4所示的表格来表示。

The system has three main components, as indicated in Fig. 1:

(1) The S-System (Sensory System) can be represented as a set of points in a TV raster, or as a set of photocells. Each raster-point in the S-system is connected to a number of units in the A-system(Association System), to which impulses are transmitted when the S-point is "on" (illuminated). Any particular S-A connection may be either positive (carrying positive or "excitatory" signals to the A-unit) or negative (tending to inhibit or suppress activity of the A-unit). A given point in the S-raster, for example, might be positively connected to ten A-units, and negatively connected to another ten units.

(2) The A-System (Association System) performs the switching functions between input and output. Each A-unit receives impulses from a number of S-points, and transmits outputs to one or more Response Units (R-Units). The A-Units are characterized by the fixed parameter θ, the threshold value which corresponds to the algebraic sum of input pulses necessary to evoke an output, and the stochastic variable v, the "output value", which may be any physically measurable characteristic of the output pulse, such as amplitude, frequency, or delay-period. The value of an A-unit's output will vary with its history, and acts as a counter, or register for the memory-function of the system.

(3) The R-System (Response System) consists typically of a relatively small number of units, which may operate type-bars or signal lights, and which are activated when the mean or net value of the signals received from the A-System exceeds a critical level θr. In addition to printing or displaying an output signal, each response unit feeds back impulses which inhibit , or cut off the activity of all mutually exclusive R-units and the A-units which might activate them. These inhibitory feed-back connections guarantee that only one response out of a mutually exclusive set can be triggered at one time. The response circuits are designed in such a manner that if impulses arrive at two R-units simultaneously, the unit whose inputs have the greatest mean (or net) value will respond first, cutting off the other through its inhibitory connections before it can be triggered. An entire set of mutually exclusive R-units thus acts like a multi-stable flip-flop, in which only one stage can be "on" at any one time. Several such mutually exclusive R-units may exist in parallel, as, for instance, the ten alphabets necessary to provide for printing a ten letter word. In such a case, the system is organized as in Fig. 2, with a distinct A-set corresponding to each R-set.

In Fig. 3, a more detailed breakdown is shown illustrating the logical composition of a perceptron designed to produce a three-binary-digit number as an output. Each binary place in the number is represented by an R-set of two members, zero and one. In Figure 3, circles have been used to represent sets of functional units, and rectangles to represent single units. The figure illustrates the effect of inhibitory connections when the response "101" is "on". All shaded areas of the diagram would be under inhibition while this response was active. Note that the intersection of an active A-set with an inhibited A-set is not inhibited, i.e., inhibitory connections go from each R-unit only to those A-subsets which do not contribute connections to it.

The entire organization of a system of this sort can be represented by a single table, or matrix of connections between units. The system of Fig. 3, for example, might be represented by a table such as that of Fig. 4.

从这张表中，可以看出一些 系统组织的一般原则 ：

（1） 该表被划分为三个彼此独立的部分，每一部分对应一个彼此独立的响应集合。每个部分均由一个 S-A矩阵和一个 A-R矩阵构成。

（2） 对于每个响应集合的S-A连接矩阵，最好使每一列都具有由正、负连接构成的唯一排列组合。

（3） 矩阵的每一列中，正连接与负连接的数量均相等。

（4） 每一个A组仅与一个R组相连接。

（5） 每个A-R子矩阵中的每一行，都应具有由连接构成的唯一排列组合。

（6） A-R子矩阵中所有不表示兴奋性连接（excitatory connections）的元素，在A与R属于同一集合的情况下，均由从R指向A的抑制性连接（inhibitory connections）填充。

在图5中给出了典型A单元和R单元的电子线路图。这些电路能够满足该模型在逻辑上的要求。A单元的数值由一个积分器（integrator）控制。每当该单元处于活动状态时，积分器的数值便增加一个很小的增量，并由此控制输出脉冲发生器（output pulse-generator）的增益。在这样的系统中，输出脉冲的振幅等同于该单元的数值。输入线路中的气体放电管（gas tube）会在数值超过 θ 时触发A单元，前提是该门控电路（gate）没有受到来自R单元的信号而关闭。

Some general principles of organization should be clear from this table.

(1) The table is organized into three independent sections, one for every independent set of responses. Each of these sections consists of an S-A matrix and an A-R matrix.

(2) Within the matrix of S-A connections for each response set, it is desirable that each column should have a unique permutation of positive and negative connections.

(3) The numbers of positive and negative connections are the same in every column of the matrix.

(4) Each A-set connects to only one R-set.

(5) Each row of an A-R sub-matrix should have a unique permutation of connections.

All elements of an A-R sub-matrix which do not represent

(6) excitatory connections are filled in with inhibitory connections from R to A, provided the A's and R's are of the same set,

Electronic schematics are shown in Fig. 5 for a typical A-unit and an R-unit which would meet the logical requirements of the model. The value of the A-unit is controlled by an integrator, which gains a slight increment every time the unit is active, and which governs the gain of the output pulse-generator. In such a system, the amplitude of the output pulse is equivalent to the value. The gas tube in the input line triggers the A-unit when the value of θ is exceeded, provided the gate is not closed by signals from the R-units.

R单元的输入网络可以设计成对所接收到的输入进行 求和 或 求平均 。图中所示的设计仅对输入进行求和，但在实际设计中，采用 求平均值的系统 可能更为理想，尤其是在预期同时到达的输入数量较少的情况下。输入积分器的电容器两端会建立起一个电压，其建立速率与输入信号的净值（或平均值）成正比。一个比较器（comparator）测量该电压与一个固定参考值 θR 之间的差值；当该电压超过阈值时，比较器便触发响应单元（response unit）。

The input network to the R-units may be designed either to sum or to average the inputs received. The design shown merely sums the inputs, but an averaging system would be preferable, particularly if the number of simultaneous inputs expected is small. A voltage is built up across the capacitor of the input integrator at a rate proportional to the net (or mean) input value. A comparator measures the difference between this voltage and a fixed reference (θR), and triggers the response unit when the threshold voltage is exceeded.

在上一节中，我们描述了感知机的物理特性，但没有试图说明其规定的组织结构为何如此设计。本节将概述这种装置运行的基本原理，并提出一些用于评价其性能的标准。在这里，我们不打算推导相关方程，但基本方程可以参见附录Ⅰ。感知机逻辑的本质，是A组（A-set）中的一种随机过程：每当某一特定刺激与某一特定响应同时出现时，将该刺激模式连接至该响应的那一子集单元，其总体数值便获得一次增量。可以将关联系统（Association System）理解为若干彼此重叠的点集，其平均值由R单元进行测量。每一个R单元所测量的，是与其相连接的特定点集或点类中处于活动状态的点的平均值。如果向各个响应传递脉冲的A单元集合足够大，那么对于任何可能呈现的刺激模式，我们都可以相当有把握地认为，总会存在一个非零的点子集，能够向某一个特定响应传递脉冲。

在我们所描述的这类系统中，每一种可能出现的不同刺激模式，都倾向于激活一组不同的A单元。如果我们不让S点产生一定比例的负向（抑制性）脉冲，那么一个规模较大、结构复杂的模式，总会激活那些可能被其组成部分的子模式，或者被该较大模式所包含的任何发光区域激活的A单元。在这种情况下，系统将失去对复杂模式的局部与整体分别建立独立联想的能力，也无法辨别不同大小的发光区域。然而，引入来自S点的抑制性脉冲，可以保证：当呈现一个“整体”时，对某个“部分”作出响应的A单元集合中，通常会有一部分单元受到抑制。尽管如此，随机选取的两个不同刺激所激活的A单元集合仍然倾向于相互重叠；也就是说，其中会包含一定比例的共同元素，记为 Pc。

为便于表述，我们采用符号 As.r表示这样一个A单元子集：它由刺激 s 激活，并且向响应单元 r 传递脉冲。因此，A1.4表示这样一组A单元：它们由刺激模式1激活，并向响应4传递脉冲。类似地，A1,3,4,8,12表示这样一组A单元：其中每一个单元都同时受到刺激模式1和刺激模式3的激活，并且每一个单元都向响应单元4、8和12传递脉冲。因此，该集合是 A1.4 的一个子集，同时也是 A3.4、A3.8 等集合的子集。由此，如果我们考虑一个仅具有两个响应的系统，并且仅讨论两个彼此无关、随机选取的刺激模式，且每一种刺激模式都激活全部A单元的50%，那么我们预期可以发现以下比例的非零子集：

In the previous section, we have described the physical characteristics of the perceptron, without attempting to justify the prescribed organization. In this section, we will outline the basic principles on which such a device operates, and introduce some criteria for evaluating its performance. We will not attempt to derive equations in this presentation, but the basic equations will be found in Appendix I. The essence of the perceptron's logic is a stochastic process in the A-set, whereby the subset of units connecting any particular stimulus pattern to any particular response gains an increment in its aggregate value every time that that particular stimulus and response occur simultaneously. The Association System can be thought of as a number of overlapping populations of points whose mean values are measured by the R-units. Each R-unit measures the mean value of active points in that particular subset, or class, of points which is connected to it. If the sets of A-units which transmit impulses to the responses are large enough, we can be reasonably sure that there will be a non-zero subset of points transmitting impulses to any particular response for every stimulus pattern which might be presented.

In a system such as we have described, there will tend to be a different set of A-units activated by every distinct stimulus pattern which might occur. If we did not include some proportion of negative (inhibitory) impulses from the S-points, then a large, complex pattern would always activate any of the A-units which might be activated by its component sub-patterns, or by any illuminated area encompassed within the larger pattern. Under such conditions, the ability to make independent associations to parts and wholes of complex patterns, or to discriminate different sizes of illuminated areas would be lost. The inclusion of inhibitory impulses from the S-points, however, guarantees that the set of A-units responding to a "part" will generally contain some members which will be inhibited when the "whole" is presented. The sets of A-units activated by two different stimuli, selected at random, will nonetheless tend to overlap; i.e., they will contain some proportion of common elements, Pc.

It is convenient to use the notation As.r to denote the subset of A-units which is activated by Stimulus s and which transmits impulses to Response Unit r. Thus A1.4 means the set of A-units which is activated by Stimulus Pattern 1, and which transmits impulses to Response 4. Similarly, A1,3,4,8,12 means the set of A-units which are each activated by Stimulus Patterns 1 and 3, and each of which transmits to the Response Units 4, 8 and 12. This will be a subset of A1.4, and will also be a subset of A3.4, A3.8, etc.. Thus, if we consider a system with only two responses, and limit ourselves to two unrelated stimulus patterns, chosen at random, each of which activates 50% of all A-units, we would expect to find non-zero subsets in the following proportions:

在该系统中，子集 A1,2.1 和 A1,2.2 表示两个刺激所共有的单元。因此，Pc，即对其中一个刺激作出响应、同时也对第二个刺激作出响应的元素所占比例，等于 0.5。

如果呈现刺激 S1，那么子集 A1.1 和 A1.2（包括 A1,2.1、A1,2.2、A1.1,2）将被激活，并向响应1和响应2传递脉冲。由于整个系统被设计成能够产生“触发器”（flip-flop）效应，因此两个响应中的一个最终会占据优势，并抑制那些不参与自身激活的关联系统集合。因此，如果 A1.2 集合的平均值（或净值）恰好大于 A1.1 集合，那么响应2将占据主导地位，而响应1将受到抑制。这几乎会立即导致这样一种状态：只有子集 A1.2（包括 A1,2.2、A1.1,2 和 A1,2.1,2）保持活动状态，而其他所有子集均受到抑制。

只要这种状态持续存在，该子集中的元素就会不断累加其数值，而受到抑制的子集中的元素则保持不变。因此，当下一次呈现相同的刺激（S1）时，那一组由该刺激激活、并倾向于引发R2的关联系统单元（A1.2），由于在前一次呈现过程中受到选择性的“强化”，其平均值很可能已经高于整个关联系统的平均水平。同一响应R2再次发生的概率也会因此提高。由此，系统便学会了将刺激S1与响应R2联系起来，并且在每一次后续试验中，这种连接都会趋于得到进一步巩固。

如果随后呈现另一个刺激 S2，它也会倾向于激活一部分与S1相同的A单元——具体而言，就是共同集合 A1,2.1 和 A1,2.2 中的那些单元。因此，集合 A1.2 所获得的部分数值增量，会延续到集合 A2.2 中，从而产生一个略微偏向响应2的倾向。由此，两个集合之间的重叠，即它们的共同性，会导致所形成的不同联想之间产生统计上的相互作用，这种相互作用有时可能造成系统性能上的错误。如果分别响应不同刺激的集合只占全部A单元中的很小一部分，那么这种相互作用就会很弱，不同联想之间发生干扰的概率也会很低，因为随机产生的偏置倾向于彼此抵消，并且很容易被集中在正确子集中的数值增益所克服。附录Ⅱ给出了一个说明联想形成过程的实例，其中所采用的系统包含64个A单元。

在这里需要指出的是，尽管子集之间的共同性使我们付出了联想之间相互干扰这一代价，但它同时也带来了更为重要的优势。假设我们设计一个包含100个A单元的系统，每个刺激由其中10个单元响应；如果我们同时要求不同刺激所对应的集合之间不存在任何共同单元，那么系统最多只能辨别10种刺激。另一方面，如果允许预期的共同性为10%，那么系统能够识别的刺激数量上限，就等于从100个元素中每次取10个元素所形成的组合数——即总计 1.73 × 10¹² 种可能的组合。

不过，这一数字不能被视为该系统的实际容量，因为早在学习到如此数量的联想之前，系统就会达到所谓的“饱和”状态。所谓饱和，是指由于相互作用效应而导致的既有联想的预期损失，恰好与新联想的预期增益相平衡。随着系统逐渐接近这一平衡状态，在不破坏既有联想的情况下“教会”系统建立新的联想将变得越来越困难；而如果我们试图重新建立原有的联想，就会发现这一过程中又会有其他联想丢失。

因此，一个更有意义的系统容量判据，是考察这样一种概率：某个已经与两个响应中的一个建立联想的刺激，在系统达到给定程度的饱和之后，仍能保持其正确的“偏好”的概率——也就是说，在系统已经学习了一定数量的联想之后，该刺激仍然倾向于产生原先正确的响应。这一概率从形式上等价于：仅根据两个A集合的平均值，正确建立二者之间辨别的概率，并将其称为 Pd。

表1给出了一些 Pd 的计算结果。该表显示了这样一种系统中，对两个响应中的某一个作出正确偏好的概率，其中系统已经分别为每个响应学习了 Ns 个联想。该表依据附录Ⅰ中的公式（5）计算，并受该附录中所列条件的限制。NA 表示与每个R单元相连接的A单元数量。表中的数值表示概率 Pd 的正态曲线纵坐标，而不是概率本身。因此，当每个R单元连接1000个A单元，并且对于给定大小的刺激，有1%的A单元作出响应（即 Pa = 0.01）时，在系统已经分别为两个响应建立了 10⁶ 个刺激联想的情况下，只经过一次训练便作出正确辨别的概率，达到 2.23σ 水平，即正确率为 0.987。在该表中，不同响应所连接的A集合之间的共同性被设定为零，即每一个A单元只向且仅向一个响应单元传递信号。为了修正不同程度的共同性 Pcr 对不同响应所连接集合产生的影响，应将表中的数值乘以 1 − Pcr，同时应将 Ns 除以每个A单元预期的输出连接数。以图3所示的方式使R集合彼此重叠，其优势与使不同刺激对应的A集合相互重叠的优势相似：它能够处理数量极为庞大的备选响应，而代价只是正确响应概率出现相对有限的下降。

从表1可以很容易看出，这类系统具有很高的潜在效率。通过使不同响应对应的A集合彼此重叠，即允许 Pcr 大于零，可以在任意一个响应集合中纳入大量彼此互斥的响应；而通过设置多个彼此独立的响应集合，则可以构成单词，或者同时根据多个彼此独立的属性对感知对象进行分析。

粗略估计表明，如果连接参数得到适当优化，一个包含2000至3000个A单元的系统，应当能够维持一个规模可与英语语言相当的词汇库。为了最大限度地提高系统容量，并减少学习新对象所需的过程，可以通过将刺激模式置于输入摄像机视野的中央，来缩小特定刺激模式的变换集合T。通过设置一个独立的R集合，并使其反馈至一组摄像机瞄准伺服机构，系统就可以很容易地训练摄像机，使其对视野中处于周边位置的任意形态进行瞄准，而无需实际辨别这些形态的具体类别。为此，只需要学习辨别视野中不同位置是否存在刺激。例如，可以将视野左下方出现某一模式，与控制摄像机向该方向移动的响应建立联想。这样，系统便能够像人眼一样，对任何可能具有意义的模式进行“注视”（fixate），从而将其“识别学习”限制在一个相对有限的中央视野范围内，这一范围与人类视觉中的中央凹（fovea）相类似。

In this system, the subsets A1,2.1 and A1,2.2 represents the units which are common to both stimuli. Pc, the proportion of those elements responding to one stimulus which also respond to the second, is therefore equal to 0.5.

If the stimulus S1 is presented, the subsets A1.1 andA1.2 (including A1,2.1, A1,2.2 A1.1,2) will be activated and impulses will be transmitted to both Responses 1 and 2. Since the entire system is constituted to yield a "flip-flop" effect, however, one or the other response will prevail, and will suppress the association sets which do not contribute to its own excitation. Thus, if the A1.2 set happens to have a greater mean (or net) value than the set A1.1, Response No. 2 will become dominant, and Response No. 1 will be suppressed. This will lead almost immediately to a condition in which only the subset A1.2 (including A1,2.2, A1.1,2, and A1,2.1,2) remains active, all others being suppressed. For as long as this condition continues, the elements of this subset will accumulate an increment to their value, while the elements of the suppressed subsets will remain unchanged. Thus, the next time the same stimulus (S1) is presented, that subset of association units (A1.2) which is excited by this stimulus and tends to evoke R2 will probably have a mean value greater than that of the association system as a whole, due to its selective "reinforcement" on the previous presentation. The same response, R2, will, therefore, have a greater probability of occurring again. The system has thus learned to associate the stimulus S1 to R2, and it will tend to fix that connection more firmly on each successive trial.

If a different stimulus, S2, is then presented, it will tend to pick up some of the same A-units as S1, -- specifically, those units in the common set, A1,2.1 and A1,2.2. Hence, part of the value increase which was gained by the set A1.2 will carry over to the set A2.2, creating a slight bias towards Response No. 2. Thus, the overlap, or communality, between the two sets, leads to a statistical interaction between the associations formed, which may occasionally cause errors in performance. If the sets responding to different stimuli each represent a small fraction of the total number of A-units, then the interaction will be small, and the probability of interference between different associations will be slight, since the random biases introduced will tend to cancel one another and can easily be overcome by the value-gain which is concentrated in the correct subset. Appendix II contains an illustrative example of the process by which associations are formed, in a system with 64 A-units.

At this point, we should note that while the communality between subsets has cost us a penalty of interference between associations, it has gained something far more important. For if we were to design a system of, say, 100 A-units, in which sets of 10 units respond to every stimulus, and if we were to insist that there be no communality between sets, then we would be limited to a total of only ten stimuli which the system might distinguish. If, on the other hand, we admit an expected communality of 10%, the limiting number of stimuli which the system might identify becomes equal to the number of combinations of 100 things taken ten at a time -- a total of 1.73 x 1012 possible combinations. This number, however, should not be taken to indicate the practical capacity of the system, because long before this number of associations has been learned, the system will have been "saturated", in the sense that the expected loss of previously acquired associations (through interaction effects) will exactly balance the expected gain of new associations. As this equilibrium condition is approached, it will become harder and hard-er to "teach" the system new associations without knocking out old ones, and if we try to re-establish the old ones, we will find that we have lost others in the process.

A more useful criterion for the capacity of the system is the probability that a stimulus which has been associated to one of two responses will retain its proper "preference" after the system has been saturated to a given level – i.e., after some number of associations has been learned by the system. This probability is formally equivalent to the probability of establishing a correct discrimination between any two A-sets on the basis of their mean values, and will be called Pd. The results of some calculations of Pd are presented in Table 1. This table shows the probability of a correct preference for one of two responses, in a system which has learned Ns associations to each response. The table has been computed by equation 5, in Appendix I and is subject to the conditions indicated there. NA is the number of A-units connected to each R-unit. The entries in the table represent the normal curve ordinates of the probability, Pd, rather than the probabilities themselves. Thus with 1000 A-units connected to each R-unit, and a system in which 1% of the A-units respond to stimuli of a given size (i.e., Pa = .O1), the probability of making a correct discrimination with one unit of training, after 106  stimuli have been associated to each response in the system, is equal to the 2.23 sigma level, or a probability of 0.987 of being correct. In the table, the communality between A-sets for different responses has been taken to be zero, i.e., each A-unit transmits to one and only one response unit. To correct for varying degrees of communality (Pcr) between the sets connected to different responses, the figures presented should be multiplied by 1-Par, and Ng should be divided by the expected number of output connections per A-unit. The advantage of overlapping the R-sets(in the manner indicated in Fig. 3) is similar to the advantage of over-lapping the A-sets for different stimuli; it permits a vast increase in the number of alternative responses which can be handled, at the cost of a relatively unimportant degradation in the probability of a correct response.

From Table 1, the potential efficiency of a system of this sort can readily be seen. By overlapping the A-sets for different responses (i.e., permitting Pcr to be greater than zero) a great number of mutually exclusive responses can be included in any one set, and by including several independent sets, words can be built up, or percepts can be analyzed according to several independent attributes simultaneously. A crude estimate indicates that a system with two to three thousand A-units should be capable of maintaining a vocabulary comparable to the English language, if connection parameters were properly optimized. In order to maximize system capacity, and to reduce the learning process for new objects, it would be desirable to reduce the set of transformations, T, of a particular stimulus pattern by centering the pattern in the field of view of the input camera. By including an independent R-set with feedback to a set of camera-aiming servos, the system can readily be made to train the camera on any forms occuring in peripheral locations in the field, without actually discriminating the particular forms. For this purpose, it would be necessary only to learn to distinguish the presence or absence of stimuli in different locations in the field, associating the presence of a pattern in the lower left, for example, with a control response moving the camera in this direction. The system would then be able to "fixate" any pattern which might prove significant, in much the same manner as the human eye, limiting its "recognition learning" to a relatively limited central field, analogous to the fovea in human vision.