Two-Stream Architecture as a Defense against Adversarial Example

Hao Ge | Xiao-Guang Tu | Mei Xie | Zheng Ma

Abstract—The performance of deep learning on many tasks has been impressive.However,recent studies have shown that deep learning systems are vulnerable to small specifically crafted perturbations imperceptible to humans.Images with such perturbations are called adversarial examples.They have been proven to be an indisputable threat to deep neural networks (DNNs) based applications,but DNNs have yet to be fully elucidated,consequently preventing the development of efficient defenses against adversarial examples.This study proposes a two-stream architecture to protect convolutional neural networks (CNNs) from attacks by adversarial examples.Our model applies the idea of “two-stream” used in the security field.Thus,it successfully defends different kinds of attack methods because of differences in “high-resolution” and “l(fā)ow-resolution” networks in feature extraction.This study experimentally demonstrates that our two-stream architecture is difficult to be defeated with state-of-the-art attacks.Our two-stream architecture is also robust to adversarial examples built by currently known attacking algorithms.

Index Terms—Adversarial example,deep learning,neural network.

1.lntroduction

With the development of convolutional neural network (CNNs),computers can deal with many tasks,such as target classification,face recognition,and license plate recognition.In some tasks,their performance is even better than that of humans.Therefore,the number of tasks performed by computers is greater than that of humans.However,hackers attack computer systems,thereby threatening the security of computers;similarly,unscrupulous people want to benefit from finding security holes in security-sensitive fields,posing safety risks to CNN-based systems.

As for the security of neural networks or almost every classification network,many adversarial examples[1]can be generalized to mislead classification results by simply adding small perturbations to original images[2].Such adversarial examples are potential threats to a wide range of applications.For example,a “No passing” sign can be detected as a “No parking” sign by a self-driving car because of some small perturbations that humans are not aware of[3].Therefore,a robust defensive method against adversarial attacks should be developed.

Existing defense methods can be roughly divided into four categories:1) Hiding the information of a target model to increase the difficulty in generating adversarial examples,e.g.,defensive distillation[3]-[9];2) training a classifier with adversarial examples to improve its precision[2];3) removing adversarial perturbations by training a denoising autoencoder[4],[7];4) training a classifier to distinguish real images from adversarial examples[6].

However,all these methods have disadvantages.For the first category,defensive distillation does not significantly increase the robustness of neural networks[10].For categories 2 and 3,adversarial examples are needed to train defenses,so these defenses are only effective against the generating adversarial examples.For the last category,Carlini and Wagner[8]showed that these adversarial detecting methods cannot defend their C&W attack with slight changes in the loss function.

Even for some powerful defense methods,such as MagNet[6]and HGR[4],Athalye and Carlini found them ineffective several days after their publication[11].Considering these challenges,we change our mind to build a defense system in a smaller scope to avoid being easily cracked.

In this paper,we propose an efficient and effective defense method against adversarial examples.Our method is independent of the generation process of adversarial samples,because it requires only real images for training.We discuss the working mechanism of our “two-stream” method and explain why it cannot be easily attacked.

2.Motivation

InFig.1,we utilize Lucid to help us analyze the kind of errors made by the neural network during information transmission,when the input image is an “adversarial example”,which causes the final misclassification.The images from left to right inFig.1correspond to the input images and the visualization results of each neuron in 4A,4D,and 5A layers in GoogleNet.Fig.1shows that the neurons in the “highresolution” network can accurately classify the categories of local areas according to textural information among receptive fields during the classification of a real image.These correct features can be delivered layer by layer.Consequently,the classification result is correct.However,as presented in the pictures on the second row,low-layer neurons cannot accurately extract local features because of the existence of adversarial perturbations;as a result,the final classification results are affected.Thus,adversarial examples influence “high-resolution” neural networks.

A set of comparative experiments are conducted to explain the causes of errors made by the neural network from a human perspective.Fig.2is obtained by dividing the two images inFig.1into 10×10 small squares and then disturbing the arrangement of the small squares.The size of each small square inFig.2is approximately equal to the size of the receptive field of the 4A layer in GoogleNet.In other words,all the information possibly obtained by each neuron in the 4A layer is included in a small square.The disordered order prevents humans to judge the category based on the context information of each small square and look at each small square independently;thus,the angle of view of each neuron node in the 4A layer is simulated.For the picture on the left,humans can classify most of the small squares to “dog” without the outline information.However,for the picture on the right,they are unable to accurately classify these small squares into “dog”.The reason is that perturbations destroy texture features,so we cannot accurately classify the small squares in the right picture by using the texture features similar to that in the left picture.

Fig.1.Feature visualization of the “Inc-v3” network by Lucid[12]:(a) real Image,(b) MIXED4A of real image,(c) MIXED4D of real image,(d) MIXED5A of real image,(e) attacked image,(f) 4A of attacked image,(g) 4D of attacked image,and(h) 5A of attacked image.

The size of the small squares inFig.2is divided according to the size of the receptive field in the 4A layer in GoogleNet.As such,our neural network encounters the same problem in facing the adversarial examples.The change in texture features destroys the feature expression of each neuron,and these errors lead to a wrong classification result after these errors uploaded through layers.As for the disordered real image,the image on the left inFig.2can still be classified into “dog” with more than 90% confidence in GoogleNet.Therefore,the classification logic of GoogleNet,a high-resolution network,is different from that of humans who rely more on globe features for image classification.

Bakeret al.[13]elaborated that neural networks do not rely on contour information in an article entitled“Deep convolutional networks do not classify based on global object shape”.In particular,[13] combined the texture of object A with the contour of object B to test the feature that the neural network is more dependent on.The previous study supported our study.

Fig.2.Disordered real image and an adversarial example.

2.1.Problem Lead to Failures of Defending Methods

As mentioned above,many defending methods likely fail against adversarial examples.These methods cannot easily detect adversarial examples[8]and correctly classify them.However,these issues have yet to be explained.In this study,we assume that they are caused by the insufficient amount of data.From the perspective of information theory,all classification problems require a particular amount of information to support their classification results.Information originates from data involved in training.Adversarial perturbations increase the entropy of pictures,so the amount of information in such pictures reduces,thereby decreasing the amount of information for the correct classification.This decrease leads to the failure of the defending methods.Adversarial perturbations degenerate the textures of images.If we want to classify such images into the right label,the defending methods should not be dependent on texture features.To achieve this goal,we should enlarge the receptive field in CNNs.This simple method is similar to resizing an image to a smaller size.In our experiments in ImageNet,the training images are resized to 32×32 pixel to avoid interference from adversarial perturbations;as a result,the testing accuracy is less than 10%.Indeed,the amount of information in ImageNet is insufficient to support 1000 categories of classification without the texture features.According to information theory,the coding algorithm with insufficient information definitely fails.Similarly,classification tasks with insufficient information resemble making a dress with a handkerchief,but this process likely has several loopholes.

In practical work,we are not able to obtain more data to offset the lack of information,so we change our mind to build the defense system in a smaller scope to avoid being easily attacked.Therefore,we propose a defense method that is extremely difficult to break under the following constraints:1) The size of the input images should be 299×299,which is the size of the input images in GoogleNet and 2) the input images should be involved in the 10 categories in CIFAR-10.

2.2.Why is the “Two-Stream” Concept Chosen?

The “two-stream” concept has been widely used in the security-sensitive field.For example,in the communications protocol,“checksum” along with the “body part” is used to transmit and check for errors during transmission.Another example is that a safe deposit box needs the keys of a banker and a customer to open.In addition,important experiments should be successfully replicated in different laboratories to be recognized.

Moreover,our research shows that the transferability of adversarial examples is good when a target classifier is within GoogleNet,incv3,incv4,ResNet,and the networks derived from them.However,the fooling ratio is much lower when the target classifier is CapsNet[14].This phenomenon occurs because the extraction of low-level features is more likely to be affected by the size of the receptive field of low-layer neurons.The low-layer neurons of advanced classification algorithms have similar receptive fields.So,the low-level features that they extract are similar,resulting in the transferability of adversarial examples in these neural networks.However,low-layer neurons in CapsNet have a larger receptive field.Consequently,CapsNet is more robust to adversarial perturbations generated by other networks.

In our “two-stream” architecture,a “l(fā)ow-resolution” network can be considered as a network with a large receptive field for low-layer neurons to deal with high-resolution images.As such,the transferability of adversarial examples between “high-resolution” and “l(fā)ow-resolution” networks is poor.Therefore,our method is effective.

3.Our Method

Similar to the workflows of SafetyNet[15]and MagNet[6],the workflow of our “two-stream” architecture consists of two steps:A detector that rejects adversarial examples and a classifier that classifies the remaining images into the right label.InFig.3,the classification results of “high-resolution” and “l(fā)owresolution” networks are fed to the comparation algorithm that acts as a detector and classifier.The specific comparation algorithm is shown in Algorithm 1.The mapping table maps the labels in ImageNet to the labels of CIFAR-10,e.g.,n02123045,n02124075,··· → “Cat”;n02110063,n02110806,···→ “Dog”.

Fig.3.Framework of our two-stream network.

This workflow is a generic backbone,and the network used in this framework can be replaced with others.For example,Incv3 in a “high-resolution” network can be replaced with VGG16,ResNet-152,or other networks trained in ImageNet.ResNet-32 in a “l(fā)ow-resolution” network can also be replaced with NiN,AllConv,or other networks trained in CIFAR-10.Thus,the flexibility of this framework greatly increases,and attackers experience difficulty in implementing white box attacks.

Algorithm 1 is the comparation algorithm,wherep1andp2are hyperparameters that serve as thresholds,which are set to 10% and 20% in our experiments,respectively.Yindicates the labels,andPrefers to the probability of these labels.YhighandPhighdenote the label and its corresponding probability of the top-5 classification results of the “high-resolution” network;andYlowandPlowrepresent the same things of the “l(fā)owresolution” network.

Algorithm 1.Classification

Input:Xn

Output:Classification resultYn

To verify the practicality of our proposed method,we build a network of 10000 user nodes and 1 server node to simulate a real network environment.The user nodes consist of 9000 normal user nodes and 1000 adversarial user nodes.Each normal user node periodically sends a real picture to the server to request the classification result,and the adversarial user node periodically sends an adversarial example.The server node should find and add these adversarial user nodes to the blacklist to prevent them from accessing the server and return the correct classification results to the normal user nodes at the same time.To achieve this goal,we adopt the following Algorithm 2 on the server node to distinguish whether a user node should be blacklisted.We record the sources of the images inand set the confidence coefficient for each IP to CC[IP]and CC[IP]∈ [0,15].We also set the blacklist to Bl[] and record the detection result of our “twostream”network in,where“1”denotes a real image,and “0”indicates that it is not a real image.

Algorithm 2.Real or fake

Input:Images received by the server node,Imgn,and the results of our “two-stream” classifier for Imgn,Yn

Output:The result sent back to the user node for Imgn:Clsn

4.Attacking Methods

In this study,we divide the attack methods into two categories,namely,types I and II.The attacks of types I and II aim to fool high-and low-resolution networks,respectively.We evaluate our defense against four popular attacks:Universal adversarial perturbations as a type I attack,the one-pixel attack and the carlini attack as type II attacks,and the fast gradient sign method (FGSM) as a both types I and II attacks.These attacks are explained as follow.

1) FGSM.Goodfellowet al.[2]introduced this adversarial attack algorithm and developed a method to generate an adversarial example by solving the following problem:x′=x+εsign(?xLoss(x,lx)).This attack is simple yet effective.Kurakinet al.[16]described an iteration version of FGSM.For each iteration,the attack applies FGSM with a small step sizeα.After each iteration,this attack clips the updated result so that the updated image remains in theεneighborhood of the original image.However,this adversarial attack can hardly fool a black box model.To address this issue,Donget al.[17]proposed the momentum iterative fast gradient sign method (MI-FGSM) to boost adversarial attacks.

2) Universal adversarial perturbations.Following their previous work[18],Moosavi-Dezfooliet al.[19]proposed this universal adversarial attack.Unlike other methods that compute perturbations to fool a network on a single image,this method can fool a network on all images.Moreover,they showed the universal perturbations to be generalized well across different neural networks.

3) One-pixel attack.Suet al.[20]introduced this adversarial attack algorithm and generated adversarial examples by modifying one pixel only.They claimed that they successfully fooled three common deep neural networks in about 70% of the tested images.This attack method generates adversarial examples without any information about the parameter values or the gradients of the network.In our experiment,we utilize “onepixel” and “three-pixel” versions to test our method.

4) Carlini attack.Carlini and Wagner[10]introduced an attack method for CIFAR-10 and MNIST.It is the most powerful type II attack we found.

5.Evaluation

We evaluate the properties of our “two-stream” architecture on three datasets:Car196[21],fgvc-aircraft[22],and ImageNet[23].Car196[21]and fgvc-aircraft[22]are fine-grained datasets,which contain 16185 images of 196 classes of cars and 10200 images of 102 kinds of aircraft,respectively.In this study,we use these two databases to examine the defensive performance of our architecture for the “automobile”,“truck”,and“airplane” categories.We apply ImageNet[23]composed of CIFAR-10-related categories selected from the original ImageNet database,e.g.,n01582220,n01601694$→bird,and n01644373,n01644900→frog.For the 1000 categories in ImageNet,217 can be classified into the 10 categories in CIFAR-10,and 783 are labeled with “other”.

The classification results of the “high-resolution” and “l(fā)ow-resolution” networks are directly used to determine whether an image is an adversarial example,so the presence of an attack method that can affect both networks will be disastrous for our framework.An experiment is performed to test the performance of advanced attack algorithms on both networks.The experimental results are shown inTable 1,and “Nonattack data” are the control group.

For the type I attack on CIFAR-10,we resize the images from 32×32 to 299×299 so that they can be susceptible to type I attacks,such as high-resolution datasets.The process shown inFig.4is applied to achieve type II attacks on high-resolution datasets.For a type II attack in high-resolution images,we apply three steps to achieve this attack.In particular,we resize the image to 32×32,expose the resized image to a type II attack,and calculate the difference between the obtained adversarial example and the 32×32 original image.We zoom in the difference map and overlay it into the original image.

InTable 1,H-Net gives the top 5 accuracy of the “high-resolution” network.L-Net indicates the top 1 accuracy of the “l(fā)ow-resolution” network.Table 1shows that the type I attack can affect the classification result of “H-Net” only,and the type II attack can influence the classification result of “L-Net” only.In other words,neither the type I attack nor the type II attack can be effective in both networks,indicating that their misclassification results are irrelevant.Therefore,their classification results should be compared,to determine whether an input image is an adversarial example.In addition,attacking methods are not the cause of the decrease in the accuracy of L-Net while attacking CIFAR-10 with the type I attacks.In our comparative experiment,we resize the images in CIFAR-10 to 299×299 and then resize them back to 32×32.We obtain 88.9% classification results of these images.Therefore,“resizing”,not attack methods,is the one that reduces accuracy.

Table 1:Classification accuracy of “high-resolution” and “l(fā)ow-resolution” networks on adversarial examples generated by different attack methods

Fig.4.Flowchart of implementing type II attacks in high-resolution images:(a) original image,(b) resize image,(c) attacked image,and (d) result image.

Table 2shows the detection and classification results of our “two-stream” architecture.“Reject” indicates the rate at which the images are detected as adversarial examples and rejected by our “two-stream”architecture.“Right” denotes the rate at which the images are not rejected and classified with the right label,and “Wrong” indicates the rate at which the images are not rejected and classified with a wrong label.Table 2further reveals that almost all the images can be either detected as an “adversarial example” or classified into the right label.In other words,producing an example that is mislabeled and not detected as an adversarial example by the “two-stream” architecture is difficult.Luet al.[15]proposed this standard to evaluate the quality of a defense method.

Table 2:Summary of the reaction of our “two-stream” architecture on various attacks

Fig.5illustrates the experimental results of simulating a real-world network environment.Each polyline represents the proportion of a class of blacklisted user nodes.The horizontal axis shows the number of images sent by a user node,and the vertical axis represents the proportion of blacklisted user nodes.

Adversarial user nodes with strong perturbations(Incv3 and Universal) are rapidly blacklisted,and the nodes with weak perturbations have a high probability of being blacklisted.InTable 2,the classification results that return to these unshielded nodes are often the correct classification results.Therefore,it is not that our defending algorithm is not strong enough,but these attack algorithms(DeepFool and Three-pixel) are not able to change all of the classification results,so we do not need to add all of them to the blacklist.As for normal user nodes,the proportion is almost equal to 0.In the 50 times of experiments we performed,only 17 normal user nodes were added to the blacklist.Therefore,our defense algorithm performs efficiently in single images and simulated real-world network environments.

Fig.5.Proportion of user nodes being blacklisted by the server node.

6.Conclusion

We propose a “two-stream” architecture to defend against adversarial examples.We compare two kinds of networks,not two specific networks,in our “two-stream” framework and analyze the effect of adversarial perturbations on neural networks to determine the possible reason why the “two-stream” concept works.Our comparison of the classification results of the “high-resolution” and “l(fā)ow-resolution” networks reveals that our“two-stream” framework can detect adversarial examples without requiring either adversarial examples or the knowledge of the generation process.The results show that:1) The framework can be further enhanced by new datasets and new backbones;2) an attacker experiences difficulty in implementing a white box attack.Experiments also show that producing an example that is mislabeled and not detected as an adversarial example is difficult via the “two-stream” architecture.Our “two-stream” architecture provides a research idea for researchers in the field of “adversarial examples”.

Disclosures

The authors declare no conflicts of interest.