my URL with special parameter doesn't fetch and index in google , I have crawler error - seo

my URL with special parameter doesn't fetch and index in google and I have crawler error for all Urls include this parameter '#!'
mysite.com/products/دوربین/1187/view/#!/productgroup-1187/attributes-576644-2207/
and fetching in google just support this:/products/دوربین/1187/view/
mean after the URL that includes '#!' and google fetch doesn't show or know another charter after
/products/دوربین/1187/view/
this parameter used for sorting and filtering
in google URL parameter show the parameter as "_escaped_fragment_" I changed same as follows Pic
crawler error URL:mysite.com/products/%DA%AF%D9%88%D8%B4%DB%8C/1145/view/?_escaped_fragment_=/productgroup-1145/attributes-100686-2305/
enter image description here

I think you might be able to encode those characters using a website like this: http://www.url-encode-decode.com For example:
mysite.com/products/دوربین/1187/view/#!/productgroup-1187/attributes-576644-2207
becomes:
mysite.com%2Fproducts%2F%D8%AF%D9%88%D8%B1%D8%A8%DB%8C%D9%86%2F1187%2Fview%2F%23%21%2Fproductgroup-1187%2Fattributes-576644-2207
Depending on the language you are using there are library functions that can help you achieve this translation programmatically.
As you are using C#, you could try the UriBuilder class. Here's some demo code:
using System;
namespace ConsoleApplication1
{
class Program
{
static void Main(string[] args)
{
var oURL = new UriBuilder("mysite.com/products/دوربین/1187/view/#!/productgroup-1187/attributes-576644-2207");
string sEscapedURL = oURL.Uri.AbsoluteUri;
Console.WriteLine("sEscapedURL = {0}", sEscapedURL);
Console.ReadLine(); //Pause
}
}
}

Related

How to get current url in view in asp.net core 1.0

In previous versions of asp.net, we could use
#Request.Url.AbsoluteUri
But it seems it's changed. How can we do that in asp.net core 1.0?
You have to get the host and path separately.
#Context.Request.Host
#Context.Request.Path
You need scheme, host, path and queryString
#string.Format("{0}://{1}{2}{3}", Context.Request.Scheme, Context.Request.Host, Context.Request.Path, Context.Request.QueryString)
or using new C#6 feature "String interpolation"
#($"{Context.Request.Scheme}://{Context.Request.Host}{Context.Request.Path}{Context.Request.QueryString}")
You can use the extension method of Request:
Request.GetDisplayUrl()
This was apparently always possible in .net core 1.0 with Microsoft.AspNetCore.Http.Extensions, which adds extension to HttpRequest to get full URL; GetEncodedUrl.
e.g. from razor view:
#using Microsoft.AspNetCore.Http.Extensions
...
Link to myself
Since 2.0, also have relative path and query GetEncodedPathAndQuery.
Use the AbsoluteUri property of the Uri, with .Net core you have to build the Uri from request like this,
var location = new Uri($"{Request.Scheme}://{Request.Host}{Request.Path}{Request.QueryString}");
var url = location.AbsoluteUri;
e.g. if the request url is 'http://www.contoso.com/catalog/shownew.htm?date=today' this will return the same url.
You can consider to use this extension method (from Microsoft.AspNetCore.Http.Extensions namespace:
#Context.Request.GetDisplayUrl()
For some my projects i prefer more flexible solution. There are two extensions methods.
1) First method creates Uri object from incoming request data (with some variants through optional parameters).
2) Second method receives Uri object and returns string in following format (with no trailing slash): Scheme_Host_Port
public static Uri GetUri(this HttpRequest request, bool addPath = true, bool addQuery = true)
{
var uriBuilder = new UriBuilder
{
Scheme = request.Scheme,
Host = request.Host.Host,
Port = request.Host.Port.GetValueOrDefault(80),
Path = addPath ? request.Path.ToString() : default(string),
Query = addQuery ? request.QueryString.ToString() : default(string)
};
return uriBuilder.Uri;
}
public static string HostWithNoSlash(this Uri uri)
{
return uri.GetComponents(UriComponents.SchemeAndServer, UriFormat.UriEscaped);
}
Usage:
//before >> https://localhost:44304/information/about?param1=a&param2=b
Request.GetUri(addQuery: false);
//after >> https://localhost:44304/information/about
//before >> https://localhost:44304/information/about?param1=a&param2=b
new Uri("https://localhost:44304/information/about?param1=a&param2=b").GetHostWithNoSlash();
//after >> https://localhost:44304
There is a clean way to get the current URL from a Razor page or PageModel class. That is:
Url.PageLink()
Please note that I meant, the "ASP.NET Core Razor Pages", not the MVC.
I use this method when I want to print the canonical URL meta tag in the ASP.NET Core razor pages. But there is a catch. It will give you the URL which is supposed to be the right URL for that page. Let me explain.
Say, you have defined a route named "id" for your page and therefore, your URL should look like
http://example.com/product?id=34
The Url.PageLink() will give you exactly that URL as shown above.
Now, if the user adds anything extra on that URL, say,
http://example.com/product?id=34&somethingElse
Then, you will not get that "somethingElse" from this method. And that is why it is exactly good for printing canonical URL meta tag in the HTML page.
The accepted answer helped me, as did the comment for it from #padigan but if you want to include the query-string parameters as was the case for me then try this:
#Context.Request.PathBase#Context.Request.GetEncodedPathAndQuery()
And you will need to add #using Microsoft.AspNetCore.Http.Extensions in the view in order for the GetEncodedPathAndQuery() method to be available.
public string BuildAbsolute(PathString path, QueryString query = default(QueryString), FragmentString fragment = default(FragmentString))
{
var rq = httpContext.Request;
return Microsoft.AspNetCore.Http.Extensions.UriHelper.BuildAbsolute(rq.Scheme, rq.Host, rq.PathBase, path, query, fragment);
}
If you're looking to also get the port number out of the request you'll need to access it through the Request.Host property for AspNet Core.
The Request.Host property is not simply a string but, instead, an object that holds both the host domain and the port number. If the port number is specifically written out in the URL (i.e. "https://example.com:8080/path/to/resource"), then calling Request.Host will give you the host domain and the port number like so: "example.com:8080".
If you only want the value for the host domain or only want the value for the port number then you can access those properties individually (i.e. Request.Host.Host or Request.Host.Port).
var returnUrl = string.IsNullOrEmpty(Context.Request.Path) ? "~/" : $"~{Context.Request.Path.Value}{Context.Request.QueryString}";
You may want to get the URL to use it on the razor side, there is an alternative way to get the home app URL:
Url.Content("~/.....")
Example
In the following example, I wanted to generate a QR code and display it in an img tag src, because of using custom route annotation in the Action, I can't use #URL.Action so as an alternative solution I use ~ like this:
<script>
$("#imgCode").attr("src", "#(Url.Content("~/generateQr/"))"+ code);
</script>
Controller Side
[Route("/generateQr/{code}")]
...
ILSpy show how it was done in Microsoft.Owin.dll.
// Microsoft.Owin.OwinRequest
using System;
/// <summary>
/// Gets the uniform resource identifier (URI) associated with the request.
/// </summary>
/// <returns>The uniform resource identifier (URI) associated with the request.</returns>
public virtual Uri Uri => new Uri(string.Concat(Scheme, Uri.SchemeDelimiter, Host, PathBase, Path, QueryString));
I wonder why they removed this property.

Specifyng a default message for Html.ValidationMessageFor in ASP.NET MVC4

I want to display an asterisk (*) next to a text box in my form when initially displayed (GET)
Also I want to use the same view for GET/POST when errors are present) so For the GET request
I pass in an empty model such as
return View(new Person());
Later, when the form is submitted (POST), I use the data annotations, check the model state and
display the errors if any
Html.ValidationMessageFor(v => v.FirstName)
For GET request, the model state is valid and no messages, so no asterisk gets displayed.
I am trying to workaround this by checking the request type and just print asterisk.
#(HttpContext.Current.Request.HttpMethod == "GET"? "*" : Html.ValidationMessageFor(v=> v.FirstName).ToString())
The problem is that Html.ValidationMessageFor(v=> v.FirstName).ToString() is already encoded
and I want to get the raw html from Html.ValidationMessageFor(v=> v.FirstName)
Or may be there is a better way here.
1. How do you display default helpful messages (next to form fields) - such as "Please enter IP address in the nnn.nnn.nnn.nnn format) for GET requests and then display the errors if any for the post?
2. What is the best way from a razor perspective to check an if condition and write a string or the MvcHtmlString
Further to my last comment, here is how I would create that helper to be used:
public static class HtmlValidationExtensions
{
public static MvcHtmlString ValidationMessageForCustom<TModel, TProperty>(this HtmlHelper<TModel> helper, Expression<Func<TModel, TProperty>> expression, string customString)
{
var returnedString = HttpContext.Current.Request.HttpMethod == "GET" ? customString : helper.ValidationMessageFor(expression).ToString();
return MvcHtmlString.Create(returnedString);
}
}
And it would be used like this #Html.ValidationMessageForCustom(v=> v.FirstName, "Please enter IP address in the nnn.nnn.nnn.nnn format")

An interesting Restlet Attribute behavior

Using Restlet 2.1 for Java EE, I am discovering an interesting problem with its ability to handle attributes.
Suppose you have code like the following:
cmp.getDefaultHost().attach("/testpath/{attr}",SomeServerResource.class);
and on your browser you provide the following URL:
http://localhost:8100/testpath/command
then, of course, the attr attribute gets set to "command".
Unfortunately, suppose you want the attribute to be something like command/test, as in the following URL:
http://localhost:8100/testpath/command/test
or if you want to dynamically add things with different levels, like:
http://localhost:800/testpath/command/test/subsystems/network/security
in both cases the attr attribute is still set to "command"!
Is there some way in a restlet application to make an attribute that can retain the "slash", so that one can, for example, make the attr attribute be set to "command/test"? I would like to be able to just grab everything after testpath and have the entire string be the attribute.
Is this possible? Someone please advise.
For the same case I usually change the type of the variable :
Route route = cmp.getDefaultHost().attach("/testpath/{attr}",SomeServerResource.class);
route.getTemplate().getVariables().get("attr") = new Variable(Variable.TYPE_URI_PATH);
You can do this by using url encoding.
I made the following attachment in my router:
router.attach("/test/{cmd}", TestResource.class);
My test resource class looks like this, with a little help from Apache Commons Codec URLCodec
#Override
protected Representation get() {
try {
String raw = ResourceWrapper.get(this, "cmd");
String decoded = new String(URLCodec.decodeUrl(raw.getBytes()));
return ResourceWrapper.wrap(raw + " " + decoded);
} catch(Exception e) { throw new RuntimeException(e); }
}
Note my resource wrapper class is simply utility methods. The get returns the string of the url param, and the wrap returns a StringRepresentation.
Now if I do something like this:
http://127.0.0.1/test/haha/awesome
I get a 404.
Instead, I do this:
http://127.0.0.1/test/haha%2fawesome
I have URLEncoded the folder path. This results in my browser saying:
haha%2fawesome haha/awesome
The first is the raw string, the second is the result. I don't know if this is suitable for your needs as it's a simplistic example, but as long as you URLEncode your attribute, you can decode it on the other end.

What is API URL executed by Twitter4j? (Search API)

I use Twitter4J libraries to access Twitter through their Search API.
I provide such a query to Twitter4j:
Query{query='#hungergames', lang='null', locale='null', maxId=-1, rpp=100, page=-1, since='null', sinceId=241378725860618240, geocode='null', until='null', resultType='recent', nextPageQuery='null'}
and
result = twitter.search(query);
but I am not sure what URL is executes internally.
Any insights into how I can find that out?
I know Twitter API documents how I should form the URL to query something here but I want to know what URL did Twitter4J execute.
The easiest way would probably be to sniff network traffic with a tool like Wireshark.
I used the following code to replicate your query:
public static void main(String[] args) throws TwitterException {
Twitter twitter = new TwitterFactory().getInstance();
Query query = new Query("#hungergames");
query.rpp(100);
query.setSinceId(241378725860618240L);
query.setResultType(Query.RECENT);
System.out.println(query);
QueryResult result = twitter.search(query);
for (Tweet tweet : result.getTweets()) {
System.out.println(tweet.getFromUser() + ":" + tweet.getText());
}
}
The line that prints the query gives me:
Query{query='#hungergames', lang='null', locale='null', maxId=-1, rpp=100, page=-1, since='null', sinceId=241378725860618240, geocode='null', until='null', resultType='recent'}
By sniffing the network traffic I found that the code is requesting the following URL:
http://search.twitter.com/search.json?q=%23hungergames&rpp=100&since_id=241378725860618240&result_type=recent&with_twitter_user_id=true&include_entities=true

Programmatically upload synonyms to google search api using java

I'm having issues figuring out how to programmatically upload synonyms to the google search api from my server using java.
1. The Authorization: The description of how to do a server to google api is explained here. Where can I find a simple example of this using java?
2. Upload synonyms: I have created the xml to be uploaded, explained here. I am not able to see how I actually upload this to the google-api. Is there an example of how this is done?
1. The Authorization
public static String getAuthorizationToken() throws IOException, HttpException{
PostMethod method = new PostMethod("https://www.google.com/accounts/ClientLogin");
method.addParameter("accountType", "HOSTED_OR_GOOGLE");
method.addParameter("Email", "my#gmail.com");
method.addParameter("Passwd", "myPassword");
method.addParameter("service", "cprose");
method.addParameter("source", "mySource");
String response = executeMethodAsString(method);
return retrieveAuthFromResponse(response);
}
2. Upload synonym
public static String updateSynonyms(String authToken, String xml) throws HttpException, IOException{
PostMethod method = new PostMethod("http://www.google.com/cse/api/default/synonyms/abcdefg1234");
method.addRequestHeader("Content-Type", "text/xml");
method.addRequestHeader("Authorization", "GoogleLogin auth=" + authToken);
RequestEntity entitiy = new StringRequestEntity(xml, "text/xml", "utf-8");
method.setRequestEntity(entitiy);
return executeMethodAsString(method);
}