so far the forums seem to be working smooth and quick.. however I do have a question.. (perhaps I'm blind or can't find it) but where the heck is the search??

 

I must be blind too,may be disabled for now???

 

Times two????

 

They havn't figured out how to get it working yet. One thread mentions the possibility of using google for the new search. Sucks because to me, that is the most important feature of this board.

So webmins, PLEASE get search working before messing with skins and all the other fancy features. I know its not as easy as a click of a button, but I'd rather the search working then posting abilities since 99% of all question have already been answered.

 

Yeah no joke, I am trying to troubleshoot my rear defroster and no search is killing me. HELP!!!!

 

Yeah, its pretty useless to have a forum without a search function. I can understand if they didn't implement it to cut down on the server load, but you woulda thought they'd say something. I'm sure it'll be fixed in short order.

Edit:
Okay, in the mean time, just go to google, type in what you're looking for, and then put site:thirdgen.org after it. This pulls up links to the old techbb2 page, but they seem to redirect fine.

 

I was wondering why I couldn't find it. How will people respond to evey post with "do a search" now?

 

It's amazing how much you miss it when its gone. I used to search this board a couple times a day for any questions I had about my car. With the huge number of saved posts on here I could find just about anything I was looking for. I sure hope they can figure it out soon.

 

The old search was an awesome tool to fing info with...hardly ever had to post
it was all there with search.

 

I'm on another forum that uses the same software, and search is available on this software, so my guess is they will get it figured out soon.

 

I agree. The search feature on this board is the most beautiful thing in the world. I have found answers to every single question I have ever had by using that feature. PLEASE bring it back in its entirety.

 

TGO is aware of the importance of the search option and Jason is looking into ways to resolve this. If the search was turned on, it causes table locking in mysql, which makes the server load get too high to the point that people cannot access the forums. Disabling the search feature (for the time being) atleast allows the forums to be accessible until a solution can be implemented.

 

I would be happy to help the admins out with the search feature. Ours took a little bit of time to get it working on our switch to vB 3.5.4. If there is interest, feel free to email me, because I would like a search feature too.

 

Unless your forum is as large as this site (close to 3 million posts), it may not be much help. It isn't so much as getting the search feature to work in vB 3.5.4, as much as it is getting search to work with a database this large in size without causing server load issues.

 

You may need lower the maxclients. Most of time this is too high for the actual connections that a server will connect. Maxclients is the maximum concurrent connections the server will have at one time, in the same nanosecond.

If you are not using MySQL, then it will be the same setup for the optimization, but you will need to look up the specification specifications. PostgreSQL, Oracle, MsSQL are all about the same.

If you are using MySQL, then these should with optimization:
http://www.vbulletin.com/forum/showthread.php?t=121386
http://www.vbulletin.com/forum/showthread.php?t=114420

 

I don't think that the maxclients setting is what is doing it. 99% of the time, there is no backlog, the box is more than capable of handling the 80-100 queries per second that are generated.

The problem comes in when someone uses one of the following:

• Site Search
• Find posts by user
• Find threads by user
• Find users with a specific IP address

Those are the queries that are causing table locking on the `post` and `thread` tables (and every once in a while, the `user` table). When that happens, all of the normal queries start to back up and the load jumps from 1.0-1.5 to 80.0-100.0 in a matter of seconds. The server stops accepting connections once it hits maxclients (as it should) and everything locks up until the original blocking query finishes.

If anyone knows how to resolve that issue, either through manipulation of the my.cnf file, or by any other means, we would be very grateful as I would LOVE to turn the search back on.

 

Id love to see a more complex search feature.
No letter count limit.

 

Jason

have you re indexed the database? just curious.. may have some ideas but that is the first thought im thinking.. probably have but just a quick thought.

 

Oh yeah, if you haven't re-indexed, we had issues until we did. I still think it's an optimization thing, but somethings are easier than others.

 

How much memory is this box running? It is dedicated right? I'll start some research to help you out with your setup. Can you give me the specs on the server?

 

Maroon-IROC-Z, I'd like to apologize if my comment above was taken wrong. I just hope that you can understand, and you probably know if you have been in similar positions, that you hear from so many people, that sometimes it seems everyone and their brother is an expert, even those who have the work done for them. Of course there are some legit ones out there, but since no reference and not much info was given, it was hard to properly gauge.

It seems that members have been earitable due to the search issue, and I can completely understand myself, it affects me as well. However, that doesn't sanely justify for a member to sign up, hide behind a new user name, to simply irrelationally call me an idiot for statements that I never even made, in addition to calling me cocky when that was exactly just what the person was being himself. Anyone who resorts to such behavior, has some issues. I'd rather be an "idiot" than unstable

 

Yes, during a restore from a SQL dump, all indicies are rebuilt automatically. Since then, I've rebuilt the problematic ones several times, the tables are just too large to prevent table locking.

The server specs are as follows:

• Dual 2.8GHz Xeon Processors (800MHz FSB) w/ HT
• 2GB of DDR2-400 RAM
• (3) 73GB 10K RPM SCSI Drives in RAID 5
• Red Hat Enterprise Linux 3
• MySQL 4.1.18 (upgraded from the RHEL3-supported 3.23.58 with the official MySQL-supplied RPMs )
• Apache 2.0.46-56 (official RHEL3 build)
• PHP 4.3.2-26 (official RHEL3 build)

Again, any help is appreciated.

 

Jason

Have you tried re indexing the whole database? i know SQL will automaticly reindex after a dump.... but have you manually started a re index on the entire data base.. I know this will take the site down for 30 min to an hour.. but could very well help if not fix the issues.

 

After the upgrade was complete, I manually rebuilt the important indicies (see below for a list). That is what most of the downtime over the weekend was. I rebuilt them both by running "REPAIR TABLE tablename QUICK" and by dropping and then adding the indicies.

Indicies that have been rebuilt at least once:

• Fulltext title in `post`
• Index threadid in `post`
• Index userid in `post`
• Fulltext title in `thread`
• Index dateline in `thread`
• Index lastpost in `thread`
• Index pollid in `thread`
• Index postuserid in `thread`
• Index forumid in `thread`
• Index birthday in `user`
• Index birthday_search in `user`
• Index username in `user`
• Index usernameid in `user`

 

ok, what really sounds like needs done is a full re indexing of the complete database.. yes this will take the site down for maintainance for 30 min to an hr most likely.. but doing a complete reindex of the complete database very well may fix the issues.

 

No, that wasn't it. What was (yes, I figured it out) happening was that a large SELECT statement was taking a significant amount of time. An UPDATE or INSERT came behind it and had to wait because of the table lock. Because UPDATE and INSERT statements have higher priority than SELECT statements, everyone else on the board got starved.

What I did to resolve it is below:

• Turned off the site.
• Set "low_priority_updates" to "1"
• Set "ft_max_word_len" to "12" (changed from 10 to allow longer searches)
• Restarted MySQL
• Ran:
  • ALTER TABLE `techbb2`.`user` DELAY_KEY_WRITE = 1;
  • ALTER TABLE `techbb2`.`thread` DELAY_KEY_WRITE = 1;
  • ALTER TABLE `techbb2`.`post` DELAY_KEY_WRITE = 1;
  • REPAIR TABLE `techbb2`.`thread` QUICK; (to rebuild the FT index)
  • REPAIR TABLE `techbb2`.`post` QUICK; (to rebuild the FT index)
• Turned on the site.

 

right on, good job!! The search works well.. thanks for all the hard work and dedication!

 

Yay!!! (10 char)

 

... and "CREATE INDEX threadpost ON post (postid,threadid)" right now to try and combat a slow query...

 

Just out of curiousity Jason and JT are you guys just admin to get the site up and running correctly, or are you going to be full time from now on? You guys are doing an awesome job, and even though I don't like some of the new things (well mainly just the ads) I know its not your choice. Thanks for the hard work.

 

I'm more familiar with commercial RDBMS's (oracle/sql server); however if this large SELECT statement is part of the search subsystem, why is it doing table locking? can the transaction isolation level for this query be lowered, so that the table doesn't get locked in the process? If the search misses a post or two that just got created within a few seconds, that's not such a bad thing.

just a thought.

 

Am I the only one who doesn't see a search button ANYWHERE on this site??

 

search was working the last couple days. and today i can't find the search button anywhere again?

 

Jason had to disable the search feature, again, because mysql was table locking, again. I believe he has some plans tomorrow.

 

Actually I disabled it right before I left. If the it boggs the system down again then we might disable it. But I know it's important so it's back up for now.

 

Sorry Dave, I thought it was Jason since he was working on it in the afternoon.

 

No need to appologize. Jason is the one who has been working on the search features. I think he has it figured out. But with some server downtime this afternoon we have to figure out what's making that happen. Simple troubleshooting. Turn one thing off, turn another thing on till we see what works best.
I completely understand how much of a hassle it is for everyone. But what's more important, people being able to access threads quickly or to have a server go down 4 or 5 times a day.
This has been our top priority and once again I personally appoligize for any of the inconvience.

 

MySQL locks tables during a search of a full-text index so that the indicies cannot change and foul the search, there is no way that I know of to change that behavior. That, however, was the point of my issuing the "DELAY_KEY_WRITE = 1" statements I mentioned above. That was meant to allow reading and writing on the post and thread tables without actually updating the indicies. The indicies wouldn't be updated until the tables were closed (meaning that no one used the site for a second or two). Until I left for the day, it was working just fine.

Since I was out of the office yesterday when Dave took the search down, I don't know exactly what query was causing the issue, or what else was going on at the same time. Disabling the search to fix the issue may have been completely coincidental, stranger things have happened... Anyway, I'll keep an eye on it.