Showing posts with label Strings. Show all posts
Showing posts with label Strings. Show all posts

Thursday, 30 April 2020

Suffix Arrays #2 Longest Repeated Substring

Power of String


Problem Statement: here


Solution:
We have to find the longest substring occurring in the string at least K times.
The golden point is: 
EVRY SUBSTRING IS A PREFIX OF SOME SUFFIX.
So, we can quickly calculate the suffix array and LCP array of the given string.
Once, we have calculated the suffix array, we now want the substring of the maximum common prefix, but it should be present at least k times. This means, in LCP array, it should be present in at least k-1 consecutive elements. So, we can use the sliding window technique to find out the minimum in a sliding window of length k-1.
Why does the longest substring need to be present in consecutive suffix array indices??
Since we know that suffix array is nothing but lexicographically shortest suffixes array. Let's say the length of the longest substring is x, and let us represent is by L. Then our string can be represented as:
S = A₁LA₂L....Aₚ, where L is the longest substring repeating at least K times. So, when we look at the suffix array, all the suffixes which start with this L, they have to be present together in our suffix array, because a large part of it is common. But, it might be possible that we have a suffix that starts from L and shares prefix length much larger than just L. Thus, we consider the minimum of the entire window because we know in minimum, they will be sharing exactly L length, if they would have been sharing more, that would have already been included in counting for L.
Complexities:
Building Suffix Array - O(NlogN)
Building LCP Table - O(N)
Computing minimum in a sliding window - O(N)
Thus, the overall complexity is O(NlogN). We could have even used any RMQ data structure like segment trees to answer the minimum in the sliding window, but we have used the most optimal way of doing it - using a monotonic queue. I will make a separate blog post (maybe) discussing the monotonic queue and its wide applications.

The code:
#include<bits/stdc++.h> using namespace std; #include <ext/pb_ds/assoc_container.hpp> #include <ext/pb_ds/tree_policy.hpp> using namespace __gnu_pbds; #define M1 1000000007 #define M2 998244353 #define ll long long int #define pll pair<ll,ll> #define mll map<ll,ll> #define F first #define S second #define PB push_back #define mp make_pair #define lb lower_bound #define ub upper_bound #define V(a) vector<a> #define endl '\n' #define test(t) while(t--) #define PI acos(-1.0) #define rep(i,a,b) for(ll i=a;i<b;i++) #define repp(i,b,a) for(ll i=b-1;i>=a;i--) #define clr(ar, val) memset(ar, val, sizeof(ar)) #define setbits(x) __builtin_popcountll(x) #define zrobits(x) __builtin_ctzll(x) #define ps(y) fixed << setprecision(y) #define all(x) begin(x),end(x) #define allr(x) rbegin(x),rend(x) const int inf= 0x3f3f3f3f; const ll INF= 0x3f3f3f3f3f3f3f3f; const int dx[4]= { 0, -1, 0, 1 }; const int dy[4]= { -1, 0, 1, 0 }; ll floor_div(ll a, ll b) { return a / b - (((a ^ b) < 0) and a % b); } ll ceil_div(ll a, ll b){ return a / b + (((a ^ b) >= 0) and a % b); } vector<int> suffix_array_cyclic(string s){ int n = s.size(), N = n + 256; vector<int> sa(n), ra(n); for(int i = 0; i < n; i++) sa[i] = i, ra[i] = s[i]; for(int k = 0; k < n; k ? k *= 2 : k++) { vector<int> nsa(sa), nra(n), cnt(N); for(int i = 0; i < n; i++) nsa[i] = (nsa[i] - k + n) % n; for(int i = 0; i < n; i++) cnt[ra[i]]++; for(int i = 1; i < N; i++) cnt[i] += cnt[i - 1]; for(int i = n - 1; i >= 0; i--) sa[--cnt[ra[nsa[i]]]] = nsa[i]; int r = 0; for(int i = 1; i < n; i++) { if(ra[sa[i]] != ra[sa[i - 1]]) r++; else if(ra[(sa[i] + k) % n] != ra[(sa[i - 1] + k) % n]) r++; nra[sa[i]] = r; } ra = nra; } return sa; } vector<int> kasai(string s, vector<int> sa) { int n = s.size(), k = 0; vector<int> ra(n), lcp(n); for(int i = 0; i < n; i++) ra[sa[i]] = i; for(int i = 0; i < n; i++) { if(k) k--; if(ra[i] == n - 1) {k = 0; continue;} int j = sa[ra[i] + 1]; while(k < n && s[(i + k) % n] == s[(j + k) % n]) k++; lcp[ra[i]] = k; if(ra[(sa[ra[i]] + 1) % n] > ra[(sa[ra[j]] + 1) % n]) k = 0; } return lcp; } inline void INP() { #ifndef ONLINE_JUDGE freopen("input.txt","r",stdin); freopen("output.txt","w",stdout); #endif } void solve() { ll n,k; cin >> k >> n; if (k == 1){ cout << n << endl; return; } string s; cin >> s; s += '#'; vector<int> sa = suffix_array_cyclic(s); vector<int> lcp = kasai(s,sa); lcp.erase(lcp.begin()); // for(int i: lcp) cout << i << " ";cout << endl; deque<pair<int,int>> dq; int gmax = INT_MIN; rep(i,0,k-1){ while(!dq.empty() and dq.back().F >= lcp[i]){ dq.pop_back(); } dq.push_back({lcp[i],i}); } gmax = max(gmax,dq.front().F); rep(i,k-1,lcp.size()-1){ while(!dq.empty() and dq.back().F >= lcp[i]){ dq.pop_back(); } dq.push_back({lcp[i],i}); while(dq.front().S <= i-(k-1)) dq.pop_front(); gmax = max(gmax,dq.front().F); } cout << gmax << endl; } int32_t main(){ ios_base::sync_with_stdio(0);cin.tie(0);cout.tie(0); int t=1; // cin>>t; test(t){ solve(); } return 0; }


Wednesday, 29 April 2020

Suffix Arrays #1 Longest Common Substring

A Story with Strings Problem Code: SSTORY

Problem Statement: here

Solution:
This is the first problem with suffix arrays. so I will give a little boring lecture on it. So, bare with me.

Suffix Arrays

This is another data structure that operates on strings. It is a much lighter version, also known as a compressed version of suffix trees, which is itself a special case of trie data structure. 
Let us come out of the ocean. Suffix Array is a data structure that stores the suffix strings in lexicographically shortest order. Note, we do not store the strings, instead, we just store the starting index of our suffix. Using the starting index, obviously, we can create the entire suffix string.

Use cases:
We will see many different applications of suffix arrays.
Today, as you saw in the problem statement, we are finding the Longest Common Substring of given 2 strings. 

We will construct the suffix array in O(nlogn) time. You can also use O(nlog^2n) to form the suffix array. After this, we can construct the LCP table of this suffix array in O(n) time. 
The LCP table tells us the longest common prefix of suffixes of our string. This construction is widely used in many questions, which involve operations with substrings.  
The way LCP is constructed is that we look at the suffix string present at index i in our suffix array and compare it with i+1 suffix string. Then, the longest common prefix among them is the value of LCP[i].

So, in our question, we can form a single string s = s2 + '#' + s1. Please note that the senital character must have precedence less than any other value present in s1 or s2. This will make sure, that it comes at the very start of our suffix array, and we can then continue to move on from the next index.

Once we have computed the LCP table,  we know that LCP[i] gives us the longest common prefix for suffixes starting at suffix[i] and suffix[i+1]. To make sure that these suffixes belong to 2 different strings, we can check whether the index at suffix[i] is less than the length of s2 or index of suffix[i+1] less than the length of s2 and the other greater than s2. Since, suffix[i] stores the index of the start of suffix string, this is possible to do!

Also, once you implement, you will notice some failures because of some test cases. 
Try these tests then and you will figure out the mistake.
Test 01:
abcadlfghijkl
abcdlfabcde
Test 02:
abcmdef
abcdefabcdl

My code:

#include<bits/stdc++.h> using namespace std; #include <ext/pb_ds/assoc_container.hpp> #include <ext/pb_ds/tree_policy.hpp> using namespace __gnu_pbds; #define M1 1000000007 #define M2 998244353 #define ll long long int #define pll pair<ll,ll> #define mll map<ll,ll> #define F first #define S second #define PB push_back #define mp make_pair #define lb lower_bound #define ub upper_bound #define V(a) vector<a> #define endl '\n' #define test(t) while(t--) #define PI acos(-1.0) #define rep(i,a,b) for(ll i=(ll)a;i<(ll)b;i++) #define repp(i,b,a) for(ll i=(ll)(b-1);i>=(ll)a;i--) #define clr(ar, val) memset(ar, val, sizeof(ar)) #define setbits(x) __builtin_popcountll(x) #define zrobits(x) __builtin_ctzll(x) #define ps(y) fixed << setprecision(y) #define all(x) begin(x),end(x) #define allr(x) rbegin(x),rend(x) const int inf= 0x3f3f3f3f; const ll INF= 0x3f3f3f3f3f3f3f3f; const int dx[4]= { 0, -1, 0, 1 }; const int dy[4]= { -1, 0, 1, 0 }; vector<int> suffix_array_cyclic(string s){ int n = s.size(), N = n + 256; vector<int> sa(n), ra(n); for(int i = 0; i < n; i++) sa[i] = i, ra[i] = s[i]; for(int k = 0; k < n; k ? k *= 2 : k++) { vector<int> nsa(sa), nra(n), cnt(N); for(int i = 0; i < n; i++) nsa[i] = (nsa[i] - k + n) % n; for(int i = 0; i < n; i++) cnt[ra[i]]++; for(int i = 1; i < N; i++) cnt[i] += cnt[i - 1]; for(int i = n - 1; i >= 0; i--) sa[--cnt[ra[nsa[i]]]] = nsa[i]; int r = 0; for(int i = 1; i < n; i++) { if(ra[sa[i]] != ra[sa[i - 1]]) r++; else if(ra[(sa[i] + k) % n] != ra[(sa[i - 1] + k) % n]) r++; nra[sa[i]] = r; } ra = nra; } return sa; } vector<int> kasai(string s, vector<int> sa) { int n = s.size(), k = 0; vector<int> ra(n), lcp(n); for(int i = 0; i < n; i++) ra[sa[i]] = i; for(int i = 0; i < n; i++) { if(k) k--; if(ra[i] == n - 1) {k = 0; continue;} int j = sa[ra[i] + 1]; while(k < n && s[(i + k) % n] == s[(j + k) % n]) k++; lcp[ra[i]] = k; if(ra[(sa[ra[i]] + 1) % n] > ra[(sa[ra[j]] + 1) % n]) k = 0; } return lcp; } void solve() { string s1,s2; cin >> s1 >> s2; int l2 = s2.size(); string s = s2 + '#' + s1 + '$'; vector<int> v = suffix_array_cyclic(s); vector<int> lcp = kasai(s,v); int mx = 0; map<string,bool> sm; for(int i=0;i<v.size()-1;i++){ if (min(v[i],v[i+1]) < l2 and max(v[i],v[i+1])>l2){ mx = max(mx,lcp[i]); } } if (mx == 0){ cout << 0 << endl; return; } for(int i=0;i<v.size()-1;i++){ if (min(v[i],v[i+1]) < l2 and max(v[i],v[i+1])>l2){ if (lcp[i] == mx){ sm[s.substr(max(v[i],v[i+1]),mx)] = true; } } } string fans = s2.substr(0,mx); if (sm[fans]){ cout << fans << endl; cout << mx << endl; return; } for(int i=mx;i<=s2.size();i++){ fans.erase(fans.begin()); fans += s2[i]; if (sm[fans]){ cout << fans << endl; cout << mx << endl; return; } } } int32_t main(){ ios_base::sync_with_stdio(0);cin.tie(0);cout.tie(0); int t=1; // cin>>t; test(t){ solve(); } return 0; }

Time Complexity: O(nlogn)

What next??

This problem is exactly the above one, but with my approach only you will be able to pass all tests, the time restrictions are tight. Although, this is a simpler version of the problem we discussed because it only asks for the maximum size of the substring. So, I won't discuss this problem.

Want a little challenging task?
We will be discussing this problem soon. 

Sunday, 26 April 2020

Strings #1 Application of KMP

Shortest Palindrome

Problem Statement: here

Solution: 
So this problem requires us to find the shortest length of palindrome by appending characters in front of the string. Clearly, the fewer characters we append, the shortest is the length. For this purpose, we must be interested in finding the longest palindromic prefix substring of the given string. This allows us to know the minimal characters from behind which when appended to the front will give us the shortest palindrome. 
For example string = "ABCCBADEA", we know the longest palindromic prefix is "ABCCBA" and if we append "AED", to the front, we will get the shortest palindrome.

So, how to find the longest prefix palindrome of a given string?

We will use the help of the KMP failure table to calculate the longest prefix palindrome for our string. We know that the failure table keeps track of the longest prefix which is also the suffix of the given string. Exploiting this fact, we can actually construct a new string T = S*S' where S' is the reverse string of S. Once we do this, let's say S = AB where A is a palindrome of largest size and B is rest, then T = AB*B'A, clearly we see KMP can be used to find out the A here.

The code:

class Solution { public: string shortestPalindrome(string s) { int n = s.size(); string sprime(s); reverse(sprime.begin(), sprime.end()); string stot = s + "*" + sprime; int ntot = stot.size(); vector<int> f(ntot,0); for (int i = 1; i < ntot; i++) { int t = f[i - 1]; while (t > 0 && stot[i] != stot[t]) t = f[t - 1]; if (stot[i] == stot[t]) ++t; f[i] = t; } return sprime.substr(0, n - f[ntot - 1]) + s; } };

What next after learning this application? 

Try your hands on this application here: https://codeforces.com/contest/1326/problem/D2
I will discuss it's solution by tomorrow!