Huge data Excel download using PHP

Ah! It was always a problem to download huge data in excel format. Tried the best library available 'PHPExcel' but the issue persists with memory limit and time taken to generate and download.

But here is the promising solution found after spending ages searching on Google. You can find it here. The author himself says "Never run out of memory with PHPExcel again".

You need to download it and include xlsxwriter.class.php where you want to fetch the data from DB and through it on to browser or save to a particular location.

Let's get started.
Below is the logic to generate the file and store in the current directory. If you want to store is at the desired location, specify the location for the method writeToFile.

include_once("xlsxwriter.class.php");
$writer = new XLSXWriter();
$data = array(
    array('year','month','amount'),
    array('2003','1','220'),
    array('2003','2','153.5'),
);

$writer = new XLSXWriter();
$writer->writeSheet($data);
$writer->writeToFile('output.xlsx');
In case if you want to download it on the fly use the below logic.
include_once("xlsxwriter.class.php");
$writer = new XLSXWriter();
$data = array(
    array('year','month','amount'),
    array('2003','1','220'),
    array('2003','2','153.5'),
);
$filename = 'output.xlsx';

$writer = new XLSXWriter();
$writer->writeSheet($data);

header('Content-Type: application/vnd.openxmlformats-officedocument.spreadsheetml.sheet');
header('Content-Disposition: attachment;filename="' . $filename . '"');
header('Cache-Control: max-age=0');

$writer->writeToStdOut();
Through this library, you can even set the column type. Please visit the official URL for more information.

WYSIWYG super light editor with jQuery & Bootstrap

Yes! It is. Summernote is the super light WYSIWYG editor for the web. It supports most of the features tinyMCE or CKEditor does but with smallest Javascript & CSS files.

You can download Summernote from here. It is the official website for the same. It has very simple getting started tutorial. 

This is article will show you how to start with summernote and couple of fixes or plugin which you don't find on the official website.

Getting started:
If you want to download and start with, click here.
For CDN, place below snippet under head tag.


<!-- include libraries(jQuery, bootstrap) -->
<link href="http://netdna.bootstrapcdn.com/bootstrap/3.3.5/css/bootstrap.css" rel="stylesheet">
<script src="http://cdnjs.cloudflare.com/ajax/libs/jquery/3.2.1/jquery.js"></script> 
<script src="http://netdna.bootstrapcdn.com/bootstrap/3.3.5/js/bootstrap.js"></script> 

<!-- include summernote css/js -->
<link href="http://cdnjs.cloudflare.com/ajax/libs/summernote/0.8.9/summernote.css" rel="stylesheet">
<script src="http://cdnjs.cloudflare.com/ajax/libs/summernote/0.8.9/summernote.js"></script>
  
Now create a textarea and attach summernote to it.

<textarea id="Description" name="Description"></textarea>

$(document).ready(function(){
   $('#Description).summernote();
});

When you don't want to show/use all the buttons in the toolbar, then you can choose/define like below
$('#Description').summernote({toolbar: [
       // [groupName, [list of button]]
       ['style', ['bold', 'italic', 'underline', 'clear']],
       ['font', ['strikethrough', 'superscript', 'subscript']],
       ['fontsize', ['fontsize']],
       ['para', ['ul', 'ol', 'paragraph']],
       ['insert', ['picture', 'link', 'video', 'table', 'hr']]
     ]});

That's it! But, if you are using summernote in Bootstrap modal, then there is a small problem. Whenever you click on Image, Video or link buttons another modal opens up with the options and while closing the second modal it will close all the modals. Quick fix: You can add this line of code as option to summernote.
$('#Description').summernote({
  dialogsInBody: true,
  toolbar: [ 
    // toolbar code if needed
  ]
});

Summernote inserts base64 code (image data URL) for any image inserted. If you don't want to store image data URL in the text (as it will hold huge data), then you can use below code:
$('#Description').summernote({
  dialogsInBody: true,
  toolbar: [ 
    // toolbar code if needed
  ],
  callbacks: {
      onImageUpload: function(image) {
          uploadImage(image[0]);
      }
  }
});

function uploadImage(image) {
    var data = new FormData();
    data.append("file", image);
    $.ajax({
        url: SITE_URL + "admin/upload-image",
        cache: false,
        contentType: false,
        processData: false,
        data: data,
        type: "post",
        success: function(url) {
            var image = $('').attr('src', url);
            $('#Descrpition').summernote("insertNode", image[0]);
        },
        error: function(data) {
            //console.log(data);
        }
    });
}

PHP code to upload an image to any directory and return URL for the image:

if (isset($_FILES['file']['name'])) {
    if (! $_FILES['file']['error']) {
        $name = md5(rand(100, 200));
        $ext = pathinfo($_FILES['file']['name'], PATHINFO_EXTENSION);
        $filename = $name . '.' . $ext;
        $destination = $this->getMediaPath() . 'images/' . $filename;
        $location = $_FILES["file"]["tmp_name"];
        move_uploaded_file($location, $destination);
        echo 'media/images/' . $filename;
    } else {
        echo "Ooops!  Your upload triggered the following error:  " . $_FILES['file']['error'];
    }
}            

That's it...

Generate documentation for PHP Code using phpDocumentor

What is phpDocumentor?
phpDocumentor is the simple and best tool to generate API documentation for PHP source code. 


Downloading & installing phpDocumentor:
There are several dependencies to install phpDocumentor you can find them on official website i.e., here

To install phpDocumentor using PEAR type below command
  • In windows, run command prompt as administrator. Goto xampp/php folder and type
    pear channel-discover pear.phpdoc.org
    pear install phpdoc/phpDocumentor
  • In linux, open terminal and type (if install xampp for linux then goto xampp/bin folder)
    pear channel-discover pear.phpdoc.org
    pear install phpdoc/phpDocumentor
Generate API using phpDocumentor:
  • In windows, goto  phpdoc.bat folder and run phpdoc  command for source directory to destination
     phpdoc -d <source directory of php code> -t <destination directory>
  • In Linux, goto [phpdocumentor installation folder]/bin/ and run command
     phpdoc -d <source directory of php> -t <destination directory>
Note:
If in case, installation method through PEAR gives you below error. Try downloading phar and proceed
Fatal error: Uncaught TypeError: Argument 1 passed to Monolog\ErrorHandler::handleException() must be an instance of Exception...

Download .phar from here
To generate API run the below command from php directory with the location of .phar file
php phpDocumentor.phar -d <source directory of php code> -t <destination directory>

If you find any error related to Graphviz, download and install software from here. Set the environment variable for Graphviz/bin directory for 'dot' executable file.

Google like Pagination

In the beginning of my career, every technical interview has this question; "How do you write pagination script in PHP?". I think you too faced the same issue. It was very important to answer this because every application is developed to list set of information and off-course you can't show every thing in a single page. It will irritate the user to scroll to the bottom of page to read.

So, the pagination. Divide the list of items in number of pages and navigate to each by clicking on each page number displayed. At the beginning, the pagination was simple was limited to 10-15 pages, but after some time in real time the requirement has grown to 10 multiple pages. Solution was to show first few page numbers, last few page numbers and in middle show the ellipsis(...). As you navigate to any page it would show definite page numbers to both the sides.


/**
 * Pagination file
 * 
 * This file provides the google like pagination solution
 * 
 * @category Saral
 * @package Pager
 * @version  0.1
 * @since  0.1
 */

/**
 * Pager class
 *
 * Class is used generate google like pagination links
 *
 * @category Saral
 * @package Pager
 * @version Release: 0.1
 * @since 26.Nov.2013
 * @author Sailesh Jaiswal
 */
class Pager
{

    /**
     *
     * css class name for links container
     *
     * @var string
     */
    private $class_pagination = 'pagination';

    /**
     *
     * css class name for default style for links
     *
     * @var string
     */
    private $class_default = 'page';

    /**
     *
     * css class name for disabled style for links
     *
     * @var string
     */
    private $class_disabled = 'disabled';

    /**
     *
     * css class name for current/selected link style
     *
     * @var string
     */
    private $class_current = 'current';

    /**
     *
     * used to set the css classes for links
     *
     * @param string $pagination            
     * @param string $default            
     * @param string $disabled            
     * @param string $current            
     */
    function setStyles($pagination = '', $default = '', $disabled = '', $current = '')
    {
        if ($pagination != '') {
            $this->class_pagination = $pagination;
        }
        if ($default != '') {
            $this->class_default = $default;
        }
        if ($disabled != '') {
            $this->class_disabled = $disabled;
        }
        if ($current != '') {
            $this->class_current = $current;
        }
    }

    /**
     *
     * generates pagination links
     *
     * @param string $url   url of the page         
     * @param integer $current_page   current page         
     * @param integer $total_records   total records         
     * @param integer $rpp            records per page
     * @param integer $adjacents      how many pages to show both the side of the current page in case if there are plenty of pages      
     * @return string
     */
    function showLinks($url, $current_page, $total_records, $rpp = 5, $adjacents = 2)
    {
        $class_pagination = $this->class_pagination;
        $page = $this->class_default;
        $current = $this->class_current;
        $disabled = $this->class_disabled;
        
        $pagination = "";
        $prev = $current_page - 1; // previous page is page - 1
        $next = $current_page + 1; // next page is page + 1
        $lastpage = ceil($total_records / $rpp); // lastpage is = total pages / items per page, rounded up.
        $lpm1 = $lastpage - 1; // last page minus 1
        $prev = $url . '?page=' . $prev;
        if ($lastpage > 1) {
            $pagination .= "<div><ul class=\"$class_pagination\">";
            // previous button
            if ($current_page > 1)
                $pagination .= "<li><a href=\"$prev\" class=\"$page\"> << previous</a></li>";
            else
                $pagination .= "<li><span class=\"$disabled\"> << previous</span></li>";
            
            // pages
            if ($lastpage < 7 + ($adjacents * 2)) // not enough pages to bother breaking it up
            {
                for ($counter = 1; $counter <= $lastpage; $counter ++) {
                    $i = $url . '?page=' . $counter;
                    if ($counter == $current_page)
                        $pagination .= "<li class=\"$current\"><a>$counter</a></li>";
                    else
                        $pagination .= "<li><a href=\"$i\" class=\"$page\">$counter</a></li>";
                }
            } elseif ($lastpage > 5 + ($adjacents * 2)) // enough pages to hide some
            {
                // close to beginning; only hide later pages
                if ($current_page < 1 + ($adjacents * 2)) {
                    for ($counter = 1; $counter < 4 + ($adjacents * 2); $counter ++) {
                        $i = $url . '?page=' . $counter;
                        if ($counter == $current_page)
                            $pagination .= "<li class=\"$current\"><a>$counter</a></li>";
                        else
                            $pagination .= "<li><a href=\"$i\" class=\"$page\">$counter</a></li>";
                    }
                    $lpm11 = $url . '?page=' . $lpm1;
                    $lastpage1 = $url . '?page=' . $lastpage;
                    $pagination .= "<li><a href='javascript: void(0)'>...</a></li>";
                    $pagination .= "<li><a href=\"$lpm11\" class=\"$page\">$lpm1</a></li>";
                    $pagination .= "<li><a href=\"$lastpage\" class=\"$page\">$lastpage</a></li>";
                } // in middle; hide some front and some back
                elseif ($lastpage - ($adjacents * 2) > $current_page && $current_page > ($adjacents * 2)) {
                    $pagination .= "<li><a href=\"$url?page=1\" class=\"$page\">1</a></li>";
                    $pagination .= "<li><a href=\"$url?page=2\" class=\"$page\">2</a></li>";
                    $pagination .= "<li><a href='javascript: void(0)'>...</a></li>";
                    for ($counter = $current_page - $adjacents; $counter <= $current_page + $adjacents; $counter ++) {
                        $i = $url . '?page=' . $counter;
                        if ($counter == $current_page)
                            $pagination .= "<li class=\"$current\"><a>$counter</a></li>";
                        else
                            $pagination .= "<li><a href=\"$i\" class=\"$page\">$counter</a></li>";
                    }
                    $lpm11 = $url . '?page=' . $lpm1;
                    $lastpage1 = $url . '?page=' . $lastpage;
                    $pagination .= "<li><a href='javascript: void(0)'>...</a></li>";
                    $pagination .= "<li><a href=\"$lpm11\" class=\"$page\">$lpm1</a></li>";
                    $pagination .= "<li><a href=\"$lastpage1\" class=\"$page\">$lastpage</a></li>";
                } // close to end; only hide early pages
                else {
                    $pagination .= "<li><a href=\"$url?page=1\" class=\"$page\">1</a></li>";
                    $pagination .= "<li><a href=\"$url?page=2\" class=\"$page\">2</a></li>";
                    $pagination .= "<li><a href='javascript: void(0)'>...</a></li>";
                    for ($counter = $lastpage - (2 + ($adjacents * 2)); $counter <= $lastpage; $counter ++) {
                        $i = $url . '?page=' . $counter;
                        if ($counter == $current_page)
                            $pagination .= "<li class=\"$current\"><a>$counter</a></li>";
                        else
                            $pagination .= "<li><a href=\"$i\" class=\"$page\">$counter</a></li>";
                    }
                }
            }
        
            // next button
            if ($current_page < $counter - 1) {
                $next = $url . '?page=' . $next;
                $pagination .= "<li><a href=\"$next\" class=\"$page\">next >></a></li>";
            } else {
                $pagination .= "<li><span class=\"$disabled\">next >></span></li>";
            }
            $pagination .= "</ul></div>\n";
        }
        return $pagination;
    }
}

$total_records = 100; //total records that you want to show as paginated
$page = $_GET['page'];
$pager = new Pager;
echo $pager->showLinks("http://example.com/posts.php", $page, $total_records, 10, 2);

Validation and Submission of form using jQuery :: Easiest Solution

In any application, form validation and submission to server is so common and redundant; there are multiple ways of doing it. And the best and easiest way that I do use or recommend is jQuery solution. 

Most popular jQuery plugins are the best solution, they have the best solution ever. 

One, jQuery Validator written and maintained by Jorn Zaefferer, a member of the jQuery team, lead developer on the jQuery UI team and maintainer of QUnit.

Two, jQuery Form by malsup, it has various plugins and solutions. Another plugin that I like from same developer is jQuery BlockUI. (give a try).


 
 
 
 
 
Name:
Email:
Note: To avoid unexpected behavior, maintain same form and fields names and id selector values. Need further explanation, let me know.

Email already exists using jQuery

It is the common requirement for any web application either it is PHP, .net, JSP etc., which has users involved. When creating/adding user from admin or end user registration we should check whether username or email address already exists. It will help to have unique registrations to our application and avoid confusion between various users. 

I see many developers struggle with this by submitting the form to server side script and validate their from database, in case username/email address already exists they take the pain to go back to previous page or form and show the error message with other form data filled. 

Few use AJAX to send the information onblur or onkeyup or onkeydown but face difficulties to stop form submission. Here is the simple solution that would help in validating username or email address existence without much hassle. 

Solution comes from popular jQuery Validator plugin I'm using this from ages. 

HTML Code:
<html>
    <head>
        <script src="https://cdnjs.cloudflare.com/ajax/libs/jquery/3.2.1/jquery.min.js"></script>
        <script src="https://cdnjs.cloudflare.com/ajax/libs/jquery-validate/1.16.0/jquery.validate.js"></script>
        <script>
              $(function(){
                    $('#UserForm').validate({
                           rules:{
                                   Name: {required: true},
                                   EmailID: { required: true, email: true, 
                                     remote: {
                                        url: "http://localhost/user/check-email",
                                        type: "post",
                                        data: {
                                              EmailID: function() {
                                                  return $( "#EmailID" ).val();
                                              },
                                              UserID: $('#UserID').val()
                                        }
                                     }
                                 }
                           }   
                    });
              });
        </script>
    </head>
    <body>
        <form action="" id="UserForm" method="post" name="UserForm">
            Name: <input id="Name" name="Name" type="text" />
            Email ID: <input id="EmailID" name="EmailID" type="text" />
            <input id="UserID" name="UserID" type="hidden" value="0" />
            <input id="Submit" name="Submit" type="submit" value="Submit" />
        </form>
</body>
</html>

 The remote block will do the trick, it sends the current field value to server side URL "http://localhost/user/check-email". This URL can be any file URL where it sends EmailID value using POST method and there you've to write SQL query to search in DB and if exists print 'false' otherwise 'true'. (Note: don't write return true or return false, just print true or false in quotes)

We can pass UserID as zero for new insertion for update UserID can be unsigned integer so that you can check if EmailID is same for other than selected UserID.

Sorting second dimension array

We can sort the second dimension array like table, for example


$users = array(
	array('name' => 'Mr. B', 'age' => 34), 
	array('name' => 'Mr. A', 'age' => 33),
	array('name' => 'Mr. C', 'age' => 32)
);

If you want to sort the array based on the name or age, here is the solution:



function arraySortByColumn(&$arr, $col, $dir = SORT_ASC){
	$sort_col = array();
	foreach ($arr as $key => $row) {
		$sort_col[$key] = $row[$col];
	}
	array_multisort($sort_col, $dir, $arr);
}

arraySortByColumn($users, 'name', SORT_DESC);
print_r($users);

Connect MySQL Remotely, Amazon EC2 using MySQL Workbench

Login to AWS Management Console. Under the security group, add inbound rule for MySQL.

First login to EC2 instance using SSH, then login to mysql 
mysql -hlocalhost -uroot -p
provide the password. Once you are in mysql prompt
CREATE USER 'testuser'@'%' IDENTIFIED BY 'testpwd'
GRANT ALL PRIVILEGES ON *.* TO 'testuser'@'%' WITH GRANT OPTION
FLUSH PRIVILEGES
Note: Setting host to '%' may be harmful, you can set your IP address to access the MySQL from your IP address only

Now open the my.cnf file from /etc/mysql and search for bind-address, the default value will be
bind-address = 127.0.0.1
change it to
 bind-address = 0.0.0.0
That's it now go to MySQL workbench, create connection with
Host:
Username: testuser
 

MySQL Simple JOINS



Simple Example: Lets say you have a Students table, and a Lockers table.
Each student can be assigned to a locker, so there is a "Locker Number" column in the student table. More than one student could potentially be in a single locker, but especially at the beginning of the school year, you may have some incoming students without lockers and some lockers that have no students assigned.
For the sake of this example, lets say you have 100 students, 70 of which have lockers. You have a total of 50 lockers, 40 of which have at least 1 student and 10 lockers have no student.
INNER JOIN is equivalent to "show me all students with lockers".
Any students without lockers, or any lockers without students are missing.
Returns 70 rows
LEFT OUTER JOIN would be "show me all students, with their corresponding locker if they have one".
This might be a general student list, or could be used to identify students with no locker.
Returns 100 rows
RIGHT OUTER JOIN would be "show me all lockers, and the students assigned to them if there are any".
This could be used to identify lockers that have no students assigned, or lockers that have too many students.
Returns 80 rows (list of 70 students in the 40 lockers, plus the 10 lockers with no student)
FULL OUTER JOIN would be silly and probably not much use.
Something like "show me all students and all lockers, and match them up where you can"
Returns 110 rows (all 100 students, including those without lockers. Plus the 10 lockers with no student)
CROSS JOIN is also fairly silly in this scenario.
It doesn't use the linked "locker number" field in the students table, so you basically end up with a big giant list of every possible student-to-locker pairing, whether or not it actually exists.
Returns 5000 rows (100 students x 50 lockers). Could be useful (with filtering) as a starting point to match up the new students with the empty lockers.
Original here

Find relative duration of an event using PHP for given date time (seconds ago, minutes ago, hours ago, days ago, weeks ago, months ago, years ago)

You might have seen that many of the live applications shows the time relative to when it was posted as seconds ago, minutes ago, hours ago, days ago, weeks ago, month ago, year ago and so on. Instead of showing the whole date and time of any possible action on the application it is shown with smaller units, more simpler ones like a minute ago, 15 minutes and 10 seconds ago etc. Here is how we achieve it using PHP, below are two methods that can be used, it requires a Date Time string in YYYY-MM-DD HH:II:SS format.


Method One:

  /**
     * returns the time ago in string
     * @param string $date_time
     *
     * @return string
     */
    function timeAgo($date_time)
    {
        $time_ago = strtotime($date_time);
        $cur_time = time();
        $time_elapsed = $cur_time - $time_ago;
        $seconds = $time_elapsed;
        $minutes = round($time_elapsed / 60);
        $hours = round($time_elapsed / 3600);
        $days = round($time_elapsed / 86400);
        $weeks = round($time_elapsed / 604800);
        $months = round($time_elapsed / 2600640);
        $years = round($time_elapsed / 31207680);
        $str = ''; // Seconds
        if ($seconds <= 60) {
            $str = "few seconds ago";
        } // Minuteselse
        if ($minutes <= 60) {
            if ($minutes == 1) {
                $str = "one minute ago";
            } else {
                $str = "$minutes minutes ago";
            }
        } // Hours
        else if ($hours <= 24) {
            if ($hours == 1) {
                $str = "an hour ago";
            } else {
                $str = "$hours hours ago";
            }
        } // Days
        else if ($days <= 7) {
            if ($days == 1) {
                $str = "yesterday";
            } else {
                $str = "$days days ago";
            }
        } // Weeks
        else if ($weeks <= 4.3) {
            if ($weeks == 1) {
                $str = "a week ago";
            } else {
                $str = "$weeks weeks ago";
            }
        } // Months
        else if ($months <= 12) {
            if ($months == 1) {
                $str = "a month ago";
            } else {
                $str = "$months months ago";
            }
        } // Years
        else {
            if ($years == 1) {
                $str = "one year ago";
            } else {
                $str = "$years years ago";
            }
        }
        return $str;
    }
ex:
echo timeAgo('2015-01-17 14:20:00'); 
o/p: 31 minutes ago

Method Two: in detail

  /**
     * gives time ago in detailed
     *
     * @param string $date_time
     * @return string
     */
    function timeAgo($date_time)
    {
        $then = new DateTime($date_time);
        $now = new DateTime();
        $delta = $now->diff($then);
        $quantities = array(
            'year' => $delta->y,
            'month' => $delta->m,
            'day' => $delta->d,
            'hour' => $delta->h,
            'minute' => $delta->i,
            'second' => $delta->s
        );
        $str = '';
        foreach ($quantities as $unit => $value) {
            if ($value == 0)
                continue;
            $str .= $value . ' ' . $unit;
            if ($value != 1) {
                $str .= 's';
            }
            $str .= ', ';
        }
        $str = $str == '' ? 'a moment ' : substr($str, 0, - 2);
        return $str . ' ago';
    }
    
ex:
echo timeAgo('2015-01-17 14:20:00'); 
o/p: 31 minutes, 14 seconds ago

Recovering an InnoDB table from only an .ibd file

Sometime you may need to recover a table when all you have is the .ibd file. In this case, if you try to load it into a new instance, your likely to encounter some errors about the table id not matching. And there is not really a way around this.
However, I’ve found two work-arounds for this:
Note: You will need the .ibd file and the CREATE TABLE statement for each table you want to recover using these methods.
  1. Simulate the internal InnoDB table counter. That is, create work tables (with innodb_file_per_table enabled) until you have the internal pointer of table id equal to (1 – id_of_ibd_table_you_need_to_restore). (See Method #1)
  2. Manually hex edit the .ibd file, changing the table id. (See Method #2)
*Note: There are some internal structures with this meta information, so you’ll need to dump/import that data after you get it loaded, so you avoid unpleasantries that will inevitably show their face.
Method #1 – Create work tables
  1. Start up clean/fresh instance of MySQL with innodb_file_per_table enabled. 
  2. Now, we need to find the table id that MySQL is currently set at, as well as the table id for the table we need to recover. Note:  Step 2 (2a – 2f) is simply to find the table id that is stored inside of the .ibd file. I’ve written a PHP script to determine this, so using the script can save a bunch of time. See the bottom of this page (under “Associated Files”) for the exact script.
    2a. Create a test database:
    mysql> CREATE DATABASE test1;
    mysql> USE test1;
    
    2b. Issue the create table command for the table:
    mysql> CREATE TABLE `product` (
      `PRODUCT_ID` bigint(20) unsigned NOT NULL auto_increment,
      `BRAND_ID` int(10) unsigned default NULL,
      `PRODUCT_TYPE_ID` int(10) unsigned default NULL,
      `GROUP_ID` int(10) unsigned default NULL,
      `PRODUCT_NAME` varchar(500) NOT NULL,
      `DEFAULT_EMAIL_ID` varchar(48) default NULL,
      `PRODUCT_STATUS` tinyint(1) NOT NULL,
      `CLIENT_ID` bigint(20) unsigned default NULL,
      `LAST_MODIFIED_BY` varchar(45) NOT NULL,
      `LAST_MODIFIED_DATE` datetime NOT NULL,
      PRIMARY KEY  (`PRODUCT_ID`)
      ) ENGINE=InnoDB;
    
    2c. Discard the tablespace, which will delete the newly created .ibd file:
    mysql> ALTER TABLE product DISCARD TABLESPACE;
    
    2d. Copy the pre-existing .ibd file to the datadir/test1 folder
    2e. Import this tablespace:
    mysql> ALTER TABLE product IMPORT TABLESPACE;
    
    This should produce the following error (at least this is most likely). The only way it would not is if MySQL’s current table id matched that of the preexisting ibd table id. In which case, you can now dump your table.
     
    ERROR 1030 (HY000): Got error -1 from storage engine
    
    2f. So, now to check the error log (manually). Look for the following entry:
    081010 11:47:40  InnoDB: Error: tablespace id in file
    '.test1product.ibd' is 1193, but in the InnoDB
    InnoDB: data dictionary it is 1.
    

    So, now we know the internal table id is at 1, and that of the ibd table is 1193.
  3. Clean up working database:
    3a. Manually move the ibd file from the $datadir to a safe location (as you will need this file again).

    3b. Drop this table.

    mysql> DROP TABLE product;
    Note this does not re-set the internal table counter. 
  4. You’ll need to create the number of tables you need to increase the internal table id value.
    In this case, you’d create 1191 test InnoDB tables (already at 1, and need to leave 1 for the actual table, so 1193-2=1191). Run below in a loop.
    for ($1=1; $i<=1191; $1++) {
           CREATE TABLE t# (id int) ENGINE=InnoDB;
    }
    I accomplished this via a simple php script. See the bottom of this page (under "Associated Files") for the exact script. 
  5. After these are created, go ahead and drop this database and all tables (as they are not needed).
    DROP DB test1; 
  6. Now, re-perform steps 2a through 2e.
    mysql> CREATE DATABASE test1;
    mysql> USE test1;
    mysql> CREATE TABLE `product` ( ... ) ENGINE=InnoDB;
    mysql> ALTER TABLE product DISCARD TABLESPACE;
    
    <-- --="" back="" copy="" datadir="" file="" here="" ibd="" is="" original="" test1="" the="" to="" where="" you="">
    
    mysql> ALTER TABLE product IMPORT TABLESPACE;
    
    Success!
    mysql> show tables;
    +-----------------+
    | Tables_in_test1 |
    +-----------------+
    | product         |
    +-----------------+
    1 row in set (0.00 sec)
  7. Now, dump the table using mysqldump, and then you can import this to any MySQL instance. Note, you must dump this and re-import it, or you'll run into problems.

    However, it's possible to encounter crashes and/or reports of corruption in the logs.


    If this happens, try to force innodb recovery (which is most likely), and then dump the table.


    Start by setting innodb_force_recovery=1 (and try 2,3,4,5,6) until the dump works.

    For this example table, I had to set innodb_force_recovery=5 before the dump would succeed.

    The # in the output file name is the value I had innodb_force_recovery set to when trying to perform the dump:
C:Program FilesMySQLmysql-5.0.68bin>
mysqldump -uroot -P3385 test1 product > product_dump1.txt
mysqldump: Couldn't execute 'show table status like 'product'':
Lost connection to MySQL server during query (2013)

C:Program FilesMySQLmysql-5.0.68bin>
mysqldump -uroot -P3385 test1 product > product_dump2.txt
mysqldump: Couldn't execute 'show table status like 'product'':
Lost connection to MySQL server during query (2013)

C:Program FilesMySQLmysql-5.0.68bin>
mysqldump -uroot -P3385 test1 product > product_dump3.txt
mysqldump: Couldn't execute 'show table status like 'product'':
Lost connection to MySQL server during query (2013)

C:Program FilesMySQLmysql-5.0.68bin>
mysqldump -uroot -P3385 test1 product > product_dump4.txt
mysqldump: Couldn't execute 'SELECT /*!40001 SQL_NO_CACHE */
* FROM `product`': Lost connection to MySQL server during
query (2013)

C:Program FilesMySQLmysql-5.0.68bin>
mysqldump -uroot -P3385 test1 product > product_dump5.txt

C:Program FilesMySQLmysql-5.0.68bin>
mysqladmin -u root -P 3385 shutdown

C:Program FilesMySQLmysql-5.0.68bin>
mysqldump -uroot -P3385 test1 product > product_dump6.txt

In fact, in this case, I could have simply started with 5.  
This is because the error log stated this:InnoDB: Error: trying to access update undo rec field 19
in index PRIMARY of table test1/product
InnoDB: but index has only 12 fields
So, I knew there was a problem trying to look at the undo logs, and from the manual, a setting of 5 says this:
"Do not look at undo logs when starting the database: InnoDB treats even incomplete transactions as committed"
However, it's best to start at 1 and work your way forward so as to prevent as much data loss as possible.
Method #2 - Hex Edit .ibd file
First of all, you'll need to backup everything (ibdata files, ib_logfile(s), data). I'd also perform a mysqldump of everything you currently have, just so you have a mysqldump of it in the event that you need it.
Also, very important, be sure to make a copy of the .ibd file for the specific table in question.
Lastly, get a copy of the CREATE TABLE statement that will recreate this table.
Then, you'll follow the steps #1-5 (but do not perform step #6 yet) outlined on the following page:
http://dev.mysql.com/doc/refman/5.0/en/adding-and-removing.html
Let me post them here for completeness, however:
  1. Use mysqldump to dump all your InnoDB tables.
  2. Stop the server.
  3. Remove all the existing tablespace files, including the ibdata and ib_log files. If you want to keep a backup copy of the information, then copy all the ib* files to another location before the removing the files in your MySQL installation.
  4. Remove any .frm files for InnoDB tables.
  5. Configure a new tablespace.
  6. Restart the server.
  7. Import the dump files.
At this point, MySQL should be running fine with an empty slate (and should have just re-created your new ibdata and log files).
Now, you'll want to recreate the table (just using the CREATE TABLE output from above), and its database to hold it.
Then, you'll basically be following the steps #1-3 outlined on the following page:
http://dev.mysql.com/doc/refman/5.0/en/multiple-tablespaces.html
1. Issue this ALTER TABLE statement:
ALTER TABLE tbl_name DISCARD TABLESPACE;
Caution: This statement deletes the current .ibd file.
2. Put the backup .ibd file back in the proper database directory (the one that you copied above).
3. Issue this ALTER TABLE statement:
ALTER TABLE tbl_name IMPORT TABLESPACE;
 
Everything should go smoothly until step #3 (above). More than likely, this will produce an error like the following on your console:
"Got error -1 from storage engine"

Now, if you look in the error log, you'll see something like:
"InnoDB: Error: tablespace id in file '.testt2.ibd' is 2,
but in the InnoDB data dictionary it is 1."

It would not produce the above error and would work fine if the existing table already had a tablespace id of 1. However, this is unlikely.
So, assuming you see the above errors, then you can modify the tablespace id actual ibd file using a hex editor. I would do this on a different copy of the ibd file (other than the original, just in case).

Note that I used "Freeware Hex Editor XVI32" for Windows for this step. Start the program, and then open the .ibd file. You'll see each byte in it's own cell. You can then click on a cell, click edit, and then edit that byte. (http://www.chmaas.handshake.de/delphi/freeware/xvi32/xvi32.htm)

Now, in this file, there are 2 places where this tablespace id is located.
For me, and I assume it should be the same for you, but just look at the values to be sure, I see the tablespace id values listed at position 37 and 41 (positions 25 and 29 in hex). In the actual hex column, if you're previous tablespace id was 2, then in positions 37 and 41, you'd see 02 and 02.

(Note these positions can change. For instance, I tested on a table with an internal id of 1193. This in hex is 04A9. However, when searching the file, for the first instance of the table id, I found the '04' in position 39 and 'A9' in position 40. Then, for the second instance of the table id, the '04' was at position 43 and the 'A9' was at position 44. So, you'll have to convert the table id to hex, and then search for that value, near the beginning of the file.)

Note that this value (02) may vary depending on what your actual tablespace id is.
Then, simply modify both of those fields to 01, and save the file.
Then, re-do the following 3 steps:
1. ALTER TABLE tbl_name DISCARD TABLESPACE;
2. Put the newly saved .ibd file back in the proper database directory
3. ALTER TABLE tbl_name IMPORT TABLESPACE;

This time, step #3 works fine.
It is at this point you should dump/import the data. At least, get a good mysqldump of this table now. You may find that this causes corruption in InnoDB, and you may need to start MySQL using --force-innodb-recovery.
http://dev.mysql.com/doc/refman/5.0/en/forcing-recovery.html Forcing InnoDB Recovery
Associated Files :: PHP Scripts
Simple PHP script - Used to create a number of InnoDB tables (to increase internal table id counter):
$dbhost = "localhost:3385";
$dbname = "test1";
$dbuser = "root";
$dbpwd  = "";

mysql_connect($dbhost,$dbuser,$dbpwd) or die(mysql_error());

for ($i = 1033; $i <= 1190; $i++) {
   $dbquery = "CREATE TABLE test1.t" . $i . " (id int) ENGINE=InnoDB";

   echo "" . $dbquery . "";

      $result = mysql_db_query($dbname,$dbquery) or die(mysql_error());

      $j = 0;

      while($row = mysql_fetch_array($result)) {
         $j++;
         echo $row[0];
      }
}

mysql_close();
PHP Internal Table ID Finder - Used to determine the internal Table ID from the binary .ibd file:
/*
Tested with tables from 4.1.23, 5.0.68, 5.1.28, and 6.0.7.
*/

// Set the filename
$filename = "C:\Users\Chris\Desktop\mysql\working\ibds\z1.ibd";

// Read 2 bytes in at a time
$offset = 2;

// Echo filename and path
echo "filename = $filename

";

// Open the filename - need 'rb' for binary file on Windows
$handle = fopen($filename, "rb");

// Define redundant, local variables for possible later functionality and/or checks
$ibd_id_bin = 0;
$ibd_id_hex = 0;
$ibd_id_dec = 0;
$ibd_id_bin2 = 0;
$ibd_id_hex2 = 0;
$ibd_id_dec2 = 0;

// Find the filesize (note: below command messes up script)
//$filesize = filesize($filename));

// Only loop through first 21 bytes - as table is is in $array[18] and $array[20]
for ($z = 0; $z <= 20; $z++) {

 // Set variable $contents equal to 2 ($offset) bytes of binary data
 $contents = fread($handle, $offset);

 // Convert $contents from binary data to hex data
 $contents2 = bin2hex($contents);

 // Convert $contents2 from hex data to decimal data
 $contents3 = hexdec($contents2);

 // Debug Output
 //echo "contents[$z] = " . $contents . "";
 //echo "contents2[$z] = " . $contents2 . "

";
 //echo "contents3[$z] = " . $contents3 . "

";

 // If position 19, array position [18], then store the values
 if ($z == 18) {
  $ibd_id_bin = $contents;
  $ibd_id_hex = $contents2;
  $ibd_id_dec = $contents3;
 }

 // If position 21, array position [20], then store the values
 if ($z == 20) {
  $ibd_id_bin2 = $contents;
  $ibd_id_hex2 = $contents2;
  $ibd_id_dec2 = $contents3;
 }
}
fclose($handle);

// More Debug output
//echo "

The table id is $ibd_id_dec

";
//echo "

The table id is $ibd_id_dec2

";

// Check to see if both values are equal.  If so, then it's
// most certain this is the correct value.
// If not, then there's a chance the positions are off for
// this table (due to versions, etc.).
if ($ibd_id_dec == $ibd_id_dec2) {
 echo "

The table id is $ibd_id_dec

";
} else {
 echo "The values from positions [18] and [20] did not match,";
             echo "so please enable debug output, and check for proper positions.";
}
 
I found this post really helpful. Thank you Chris Calendar.
Original post: http://www.chriscalender.com/tag/innodb-error-tablespace-id-in-file/

Convert Date Time to GMT/UTC

If you want to convert any date time to GMT/UTC for given timezone (observing DST/not).

/**
     * coverts the given date time to GMT/UTC based on timezone provided
     * @param string $date
     * @param string $timezone
     * @return string
     */
    function getGMT($date, $timezone){
        date_default_timezone_set("UTC");
        $daylight_savings_offset_in_seconds = timezone_offset_get( timezone_open( $timezone ), new DateTime() );
        return $new_date = date('Y-m-d H:i:s', strtotime('-'.$daylight_savings_offset_in_seconds.' seconds', strtotime($date)));
    }



$date = "2014-11-30 23:50:00"; //yyyy-mm-dd
//$date = "11/30/2014 23:50:00"; //mm/dd/yyyy
//$date = "30-11-2014 23:50:00"; //dd-mm-yyyy
$timezone = "Asia/Kolkata";
//$timezone = "Australia/Adelaide";
//$timezone = "America/New_York";

echo getGMT($date, $timezone);