广度优先,模拟爬虫,嗅探URL

本文主要是介绍广度优先,模拟爬虫,嗅探URL，希望对大家解决编程问题提供一定的参考价值，需要的开发者们随着小编来一起学习吧！

在窗体上放两个按钮,一个RichText

using System;
using System.Collections.Generic;
using System.Windows.Forms;

namespace WindowsFormsApplication3
{
    public partial class Form1 : Form
    {
        private Dictionary<string, int> URLs = new Dictionary<string, int>();
       
  
        public Form1()
        {
            InitializeComponent();
        }

        private void button1_Click(object sender, EventArgs e)
        {
            button2.Enabled = false;
            long T1=DateTime.Now.Ticks;
            URLs.Clear();
            URLs.Add("http://www.Csdn.Net/default.aspx", 0);

            //最大允许深度5
            for (int i = 0; i < 5; i++)
            {
                widthFirst(i);
            }
            long T2=DateTime.Now.Ticks;
            MessageBox.Show("完成" + URLs.Count + "耗时：" + TimeSpan.FromTicks(T2-T1).Milliseconds + "毫秒");
            button2.Enabled = true;
           
        }
        private void widthFirst(int Deep)
        {
            Dictionary<string, int> tempURLs = new Dictionary<string, int>();
            foreach (string url in URLs.Keys)
            {
                if (URLs[url] == Deep)
                {
                    string[] urlArr = getUrl(url);
                    foreach (string newUrl in urlArr)
                    {
                        if (!URLs.ContainsKey(newUrl) && !tempURLs.ContainsKey(newUrl))
                        {
                            tempURLs.Add(newUrl, Deep + 1);
                        }
                    }
                }
            }
            foreach (string newUrl in tempURLs.Keys)
            {
                URLs.Add(newUrl, tempURLs[newUrl]);
            }  
        }

        /// <summary>
        /// 模拟得到地址数组
        /// </summary>
        /// <returns></returns>
        private string[] getUrl(string url)
        {
            Random Rnd = new Random();
            string[] Arr = new string[Rnd.Next(2,10)];            
            for (int i = 0; i < Arr.Length; i++)
            {
                //如果模拟的地址容易冲突就可能产生只搜索2层的事情。
                Arr[i] = "http://www.Csdn.Net/" + Guid.NewGuid().ToString() + ".aspx";
            }
            return Arr;
        }

        private void button2_Click(object sender, EventArgs e)
        {
            int i = 0;
            foreach (string url in URLs.Keys)
            {
                i++;
                this.richTextBox1.Text += i.ToString()+"第" + URLs[url] + "层：" + url + "/n";
                Application.DoEvents();
            }
        }
    }
}

这篇关于广度优先,模拟爬虫,嗅探URL的文章就介绍到这儿，希望我们推荐的文章对编程师们有所帮助！

---