FAQ Database Discussion Community


How to convert euro (€) symbol from Windows-1252 to UTF-8?

php,encoding,utf-8,windows-1252
A software generates me a Windows-1252 XML file, and I would like to parse it in PHP, and send the data on my database in UTF8. I tried a lot of solutions, such as iconv or utf8_encode functions, but no result. It displays things like €, but not just €......

Encoding JSON in Hidden Element

javascript,json,encoding
I've inherited some code. The previous developer was cramming JSON into a hidden HTML element. It works most of the time, except for when a single quote is included in one of the text strings. I'm not sure how to fix it. For example: <input id="options" type="hidden" value='{"1":[{ "value":"101", "text":"Bill\'s"},...

How to make the output from Text::CSV utf8?

perl,csv,encoding,utf-8
I have a CSV file, say win.csv, whose text is encoded in windows-1252. First I use iconv to make it in utf8. $iconv -o test.csv -f windows-1252 -t utf-8 win.csv Then I read the converted CSV file with the following Perl script (utfcsv.pl). #!/usr/bin/perl use utf8; use Text::CSV; use Encode::Detect::Detector;...

The File Encoding Is utf8 but is in Windows-1256 readable

encoding,utf-8
I am working on files with unknown encoding at first but I get the encoding with this lines in JAVA: InputStream in = new FileInputStream(new File("D:\\lbl2\\1 (26).LBL")); InputStreamReader inputStreamReader = new InputStreamReader(in); System.out.print(inputStreamReader.getEncoding()); and we get UTF8 in output. but the problem is that when I try to see file...

XML string to xml unable to switch encoding

sql-server,xml,sql-server-2005,encoding
I have an XML string that is malformed. DECLARE @xmlt TABLE(xstr nvarchar(max), xml xml) INSERT INTO @xmlt(xstr) VALUES ( ' <?xml version="1.0" encoding="windows-1257" ?> - <objects><object id="778913">a</object> - <object id="785491">b</object> - <object ...goes on... - </objects> ' To be able to use that XML I convert it to XML UPDATE...

Cyrllic characters in SVG font

javascript,svg,unicode,encoding,fonts
You will have to bear with me with this, as I know very little about encodings so may be asking something very simple/obvious. I am working with some SVG fonts in browser-side Javascript and I need to pull out some information on Cyrillic characters grammatically. I am doing this currently...

How can I set the encoding of a httpExchange response?

java,encoding
I'm trying to modify some server code which uses an httpExchangeobject to handle the server's response to the client. My issue is that for responses containing characters not supported by iso-8859-1, such as Chinese characters, I get something like '????' in place of the characters. I'd like to set the...

“'charmap' codec can't encode character” (Http Request)

python,encoding,request
I am using urllib.request.urlopen to get a *.srt file from a web API. The (relevant) code (Python 3.x): with urllib.request.urlopen(req) as response: result = response.read().decode('utf-8') print(result) with open(subpath, 'w') as file: file.write(result) file.close() This works fine, with the exception of some files. With certain files I receive the following error:...

Python Encoding: Open/Read Image File, Decode Image, RE-Encode Image

python,image,encoding,character-encoding
Note: I don't know much about Encoding / Decoding, but after I ran into this problem, those words are now complete jargon to me. Question: I'm a little confused here. I was playing around with encoding/decoding images, to store an image as a TextField in a django model, looking around...

jQuery autoNumeric plugin replace euro sign with question mark

jquery,encoding
i'm using autoNumeric, a jQuery plugin that automatically formats currency. Everything works fine, but the euro sign (€) is replaced by question mark (�). I have already read this and this, but it not seems to be an encoding problem couse I'm using the following meta tags on the page:...

What's the difference between CGI.unescape and URI.decode_www_form_component?

ruby,encoding
These functions seem to do the same thing. irb> CGI.unescape "Sloths%3A+Society+and+Habitat" => "Sloths: Society and Habitat" irb> URI.decode_www_form_component "Sloths%3A+Society+and+Habitat" => "Sloths: Society and Habitat" What's the difference?...

Ruby incompatible character encodings

ruby,encoding,watir,watir-webdriver
I am currently trying to write a script that iterates over an input file and checks data on a website. If it finds the new data, it prints out to the terminal that it passes, if it doesn't it tells me it fails. And vice versa for deleted data. It...

python encode the defined variable from txt file

python,python-3.x,encoding
how to get the the text itself from a txt file in python 3? so, I've a file that contains string like that: "name":"\u0414\u043e\u0436\u0434\u044c" The code I wrote is: with open(r'C:\Temp\f1.txt','r') as f1: data=f1.read() f1.close() #print(data) f2 = open(r'C:\Temp\f2.txt','w+',encoding='utf-8') f2.write(data) f2.close() I wonder why the data var is not 'utf-8'...

Haskell: quoteFile fails on text file with “invalid byte sequence” on unicode characters

linux,haskell,unicode,encoding,utf-8
I'm facing issue with quoteFile in my virtual environment (Debian Wheezy with GHC 7.8.4 installed). I have described file oriented version of st quasi quoter from Text.Shakespeare.Text: import Language.Haskell.TH.Quote (QuasiQuoter, quoteFile) import Text.Shakespeare.Text (st) sfFile :: QuasiQuoter stFile = quoteFile st This works very well on my host machine, however,...

Python 3 not aware of Windows filename encodings?

windows,python-3.x,encoding,path,filenames
The following code works well in Win7 until it crashes in the last print(f). It does it when it finds some "exotic" characters in the filenames, as the french "oe" as in œuvre and the C in Karel Čapek. The program crashes with an Encoding error, saying the character x...

UTF-8 with php pdo and mysql - ctype php

php,mysql,pdo,encoding,utf-8
How to use ctype_alpha with UTF-8 I have this code: if(empty($_POST) === false) { if (isset($_POST['first_name']) && !empty ($_POST['first_name'])){ if (ctype_alpha($_POST['first_name']) === false) { $errors[] = 'Please enter your First Name with only letters!'; } } so here I chech is everzthing fine if not I got error... But If...

Represent string as an integer in python

python,string,encoding
I would like to be able to represent any string as a unique integer (means every integer in the world could mean only one string, and a certain string would result constantly in the same integer). The obvious point is, that's how the computer works, representing the string 'Hello' (for...

Reading file containing Arabic language

c#,file,encoding,printing,character-encoding
I have a file that contains both Arabic and English word/letters/numbers. I'm trying to print the file using the code from Here. When I open the file in notepad, I see all the funny and unprintable chars. When I save the same file as Unicode in Save as... file menu...

store Hebrew in database with utf- 8 encoding using php and mysql

html,mysql,internet-explorer,encoding,utf-8
I am building a web site with database in Hebrew (using php and mysql). on general I use chrome to test my job... today I tried explorer :-0 (yeh... you must be thinking "why the f*** would he do it... ;) ). I found out that all the data that...

HTML instead of URI encoding in regard to XSS?

html,encoding,uri,xss
I'm inserting untrusted data into a href attribute of an tag. Based on the OWASP XSS Prevention Cheat Sheet, I should URI encode the untrusted data before inserting it into the href attribute. But would HTML encoding also prevent XSS in this case? I know that it's an URI context...

Why Ant add an equal character in properties file

encoding,ant,properties-file
I have a some given properties file like this : firstvalue=1 secondvalue=hello I use this Ant script to change a value : <propertyfile file="A.properties"> <entry key="firstvalue" value="2" /> </propertyfile> After executing the script, the properties file is now : #Wed, 20 May 2015 11:42:46 +0200 = firstvalue=2 secondvalue=hello I understand...

Puppet Windows Package Provider Encoding Error

ruby,windows,encoding,puppet
I have a Puppet manifest for Windows that is used (among other things) to download and install Python. It used to work just fine. Now it's giving me strange errors related to string encoding whenever it tries to process an MSI file. The relevant portions of the manifest are: (Note:...

Delete weird ANSI character and convert accented ones using Python

python,encoding,utf-8,ansi
I've downloaded a bunch of Spanish tweets using the Twitter API, but some of them have strange ANSI characters that I don't want there. I have around 18000 files and I want to remove those characters. I have all my files encoded as UTF-8. For example: b'Me quedo con una...

Why does opening a file in two different encodings work as expected?

python,python-3.x,encoding,character-encoding,text-files
Quoting from here, The default encoding is platform-dependent, so this code might work on your computer (if your default encoding is utf-8), but then it will fail when you distribute it to someone else (whose default encoding is different, like CP-1252). Code mentioned in the above quote: fp = open('text.txt')...

Enforcing that inputs sum to 1 and are contained in the unit interval in scikit-learn

python,numpy,encoding,machine-learning,scikit-learn
I have three inputs: x=(A, B, C); and an output y. It needs to be the case that A+B+C=1 and 0<=A<=1, 0<=B<=1, 0<=C<=1. I want to find the x that maximizes y. My approach is to use a regression routine in scikit-learn to train a model f on my inputs...

Is there any function like iconv in Python?

php,python,encoding
I am new on Python. I have some CSV files need to convert from shift-jis to utf-8. Here is my code in PHP, which is successful transcode to readable text. $str = utf8_decode($str); $str = iconv('shift-jis', 'utf-8'. '//TRANSLIT', $str); echo $str; My problem is how to do same thing in...

How can we create our own string encoding-decoding or encryption-decryption script in java without using any given library i.e. Base64, AES, etc?

java,security,encryption,encoding,cryptography
How can we create our own string encoding-decoding or encryption-decryption script in java without using any given library i.e. Base64, AES, etc? As per my client requirements, he want something where a user can decide a particular letter/number to be replaced by a letter/number which is again choosed by the...

Different output when run on Azure than on local build

c#,asp.net-mvc,azure,encoding
I have a C#, .NET MVC 4.5 web app with an action method which calls out to a web service with a query string parameter. Something like "[email protected]" I'm using Encoding.UTF8 and my return code looks like this: return Content(responseText); This outputs as: 6^.a308666-86 which is correct. However, when I...

Getting hebrew json to php file

php,json,encoding,hebrew
I'm trying to parse this json file in php: http://www.oref.org.il/WarningMessages/alerts.json This file contains Hebrew letters which cause encoding problems. My script: $content = file_get_contents('http://www.oref.org.il/WarningMessages/alerts.json'); $json = json_decode($content); echo $json->id; It just won't display anything. I just get a blank page. But if I do echo $content; it shows the json...

Encode UTF-8 for list

python,python-2.7,encoding,utf-8
I'm using selenium to retrieve a list from a javascript object. search_reply = driver.find_element_by_class_name("ac_results") When trying to write to csv, I get this error: Traceback (most recent call last): File "insref_lookup15.py", line 54, in <module> wr_insref.writerow(instrument_name) UnicodeEncodeError: 'ascii' codec can't encode character u'\xe4' in position 22: ordinal not in range(128)...

Python encoding issue through sockets

python,sockets,encoding,utf-8
I am creating a short python script which requires me to send text back and forth between two computers via sockets. Now when I try to test my application locally on the same computer using telnet and it requires me to use the 'utf-8' encoding when I use the bytes()...

Encoding problems in Python - 'ascii' codec can't encode character '\xe3' when using UTF-8

python,encoding,utf-8
I've created a program to print out some html content. My source file is in utf-8, the server's terminal is in utf-8, and I also use: out = out.encode('utf8') to make sure, the character chain is in utf8. Despite all that, when I use some characters like "ã", "é" in...

node.js - issue with Arabic character encoding

json,node.js,encoding,arabic
I'm using node.js and express to send a Json file to client. the String in the database is something like "السلام". but the output of Json in the browser looks like this "های پاتایا". now the Jquery(or maybe something else) in the browser correctly decodes this to arabic, but in...

Why does this fail to decode a base64 string? [closed]

java,encoding,base
I have created some code that should work: String input = "PONT4uYmTYwiP0omcAxZG8a3vKI="; String expectedOut = "3ce353e2e6264d8c223f4a26700c591bc6b7bca2"; Base64.Decoder decoder = Base64.getDecoder(); byte[] outAsByte = decoder.decode(input); String output = new String(outAsByte, StandardCharsets.UTF_8); System.out.println(input); System.out.println(output); (based on How do I decode a base64 encoded string? - I know that it's in C#) Surprisingly...

Reading file data using HttpWebResponse

c#,.net,web-services,encoding
I am attempting to read a file from a web service. The file is encrypted and the decryption function expects the input (cipher text) as a base 64 encoded string. The following code executes but I am unable to decrypt the resulting string. var request = (HttpWebRequest) WebRequest.Create(new Uri(_targetUrl)); request.Method...

Downloading youtube mp3 - metadata encoding issue (python, youtube-dl, ffmpeg)

python,encoding,ffmpeg,metadata
I'm trying to download audio from youtube with youtube-dl.exe and ffmpeg.exe (Windows 7), but I am having some troubles with encoding. I have to parse metadata manually, because when I try to use --metadata-from-title "%(artist) - %(title)" --extract-audio --audio-format mp3 https://www.youtube.com/watch?v=DaU94Ld3fuM I get ERROR: Could not interpret title of video...

Downloaded octet-stream then encoding as pdf; can't get line endings worked out

pdf,encoding,httpresponse,fiddler
Tools that I'm using for this: Chrome Notepad++ Sublime Text 3 Fiddler WinMerge Adobe Acrobat Reader X Synopsis I have downloaded a pdf twice, once through Chrome as an experimental control; once again through a raw /GET request via Fiddler which returns me an octet-stream. To this point, I can...

Perl XML::Twig character encoding

xml,perl,encoding,utf-8
I have a set of XML files with a combination of non-simple ASCII characters and encoded characters, e.g.: ... many 8-bit characters such as é, &#10906;, and ñ. (The second character is the ampersand-semicolon version of ⪚. The first and third are the unescaped characters.) The files are in UTF-8...

tee with utf-8 encoding

powershell,encoding,tee,powershell-v4.0
I'm trying to tee a server's output to both the console and a file in Powershell 4. The file is ending up with a UTF-16 encoding, which is incompatible with some other tools I'm using. According to help tee -full: Tee-Object uses Unicode enocding when it writes to files. ......

Is a base64 encoded string unique?

encoding,base64
I can't find an answer to this. If I encode a string with Base64 will the encoded output be unique based on the string? I ask because I want to create a token which will contain user information so I need make sure the output will be unique depending on...

ctrl+G in erl doesn't work

unicode,encoding,utf-8,erlang,docker
I'm trying to interconnect erlang nodes, but entering ctrl+G doesn not work: Eshell V6.4.1 (abort with ^G) 1> ^G Eshell V6.4.1 (abort with ^G) 1> ^G Eshell V6.4.1 (abort with ^G) 1> ^G Eshell V6.4.1 (abort with ^G) any idea why this can happen? I was thinking about locale settings,...

Add an attribute to an object with different kind of accent in Angular

javascript,angularjs,encoding,character-encoding,escaping
my usecase is: I have a ng-model "myModel". I'm pulling the html for a form via ajax (because I have dynamic fields) and in it, the fields are described with: <input ng-model="myModel.myField" name="myField"> This works perfectly except when the name of the field has an accent in it (it is...

How to bencode non-ascii strings and non-integer numbers?

encoding,bencoding
According to the bencoding specification: Bencoded strings are encoded as follows: <string length encoded in base ten ASCII>:<string data>, or key:value Note that there is no constant beginning delimiter, and no ending delimiter. Example: 4:spam represents the string "spam" Example: 0: represents the empty string "" Integers are encoded as...

How to process a file in correct encoding in powershell?

mysql,powershell,encoding,utf-8,cmd
I have a MySQL's SQL file which is encoded in UTF8 called data.sql (which is produced by the mysqldump command). If I run the following in a cmd.exe console the file is correctly processed. mysql --defaults-file=mysql.update2.ini --batch --raw --database=test1 --default-character-set=utf8 < "data.sql" If I run the following in a powershell...

Fast conversion of String to byte[] using UTF-16LE encoding

java,android,optimization,encoding,character-encoding
I need to get bytes of millions of string using this : String str="blablabla...."; // some UTF-16LE encoding string extracted from DB bytes=str.getBytes("UTF-16LE") But this is awfully slow. There are custom fast versions of getBytes but they don't support UTF-16LE. For example this is one of them: // http://stackoverflow.com/questions/12239993/why-is-the-native-string-getbytes-method-slower-than-the-custom-implemented-getb private...

erlang os:cmd() command with UTF8 binary

unicode,encoding,erlang,utf
I'm trying to get an Erlang function to execute a bash command containing unicode characters. For example, I want to execute the equivalent of: touch /home/jani/ჟანიweł I put that command in variable D, for example: io:fwrite("~ts", [list_to_binary(D)]). touch /home/jani/ჟანიwełok but after I execute: os:cmd(D) I get file called á??á??á??á??weÅ?. How...

JSON in Python: encoding issue on OS X, no issue on Windows

python,json,windows,osx,encoding
I have a file log.json that contains one line: {"k":"caf\u00e9"} I run the following code on Windows 7 SP1 x64 Ultimate: import json a = json.load(open('log.json', 'r')) f = open('test.txt', 'w') f.write(a['k']) I don't have any issue. When I run the same code on Max OS X 10.10 x64: Traceback...

Using UTF-8 identifier

java,android,encoding,utf-8
I get an String Stream form HTTP Request. The Stream looks like: <?xml version="1.0" encoding="utf-8"?> The first three tokens means that the String is encoded to UTF-8. I'm making Files with the String. While reading them i get an error: With this method i'm making Files with that String: private...

Can we convert ANSI encoded CSV file to utf-8 encoded file with javascript?

javascript,encoding,character-encoding
I have been looking answer for this question for days. Actually, I need to upload the ansi encoded csv file to web service. The front end application reads the file encode it's content to base64 and send it to web service in json format. But when I try to read...

gstreamer unalble to encode UYVY as h264

encoding,gstreamer
I am trying to encode a RAW stream in UYVY colorspace to H264 using gstreamer vaapi encoder. Source Pipeline: gst-launch-1.0 -e videotestsrc ! video/x-raw, format=UYVY , framerate=60/1, width=1920, height=1080 ! filesink location=raw.yuv Encoder pipeline: gst-launch-1.0 -v filesrc location=raw.yuv ! videoparse format=uyvy width=1920 height=1080 framerate=60 ! vaapiencode_h264 tune=high-compression ! mpegtsmux !...

IOS Swift Soap Message with Ampersand in value

ios,web-services,swift,soap,encoding
I have the following code to call web service from my IOS Swift application. (The code is execute when a button on the page is pressed.) The challenge is with the value for Hours. When the ampersand is in the value, I get an http error code 400. When I...

Can't get UTF-8 Special Chars to Correctly Write to MySQL (PHP)

php,mysql,encoding,utf-8
I am creating a PHP script run from the command line on a typical LAMP stack (L = OS X) and am having a lot of trouble getting special chars to record properly in the database. This script scans a directory recursively and inserts complete path into a MySQL database...

Java to JS and JS to Java encryption using cryptojs

java,encryption,encoding,aes,cryptojs
I got on this post a couple of weeks ago and worked perfectly: Compatible AES algorithm for Java and Javascript Now, I need to do the reverse operation, but once in java, I am getting this exception: javax.crypto.BadPaddingException: Given final block not properly padded at com.sun.crypto.provider.CipherCore.doFinal(CipherCore.java:966) at com.sun.crypto.provider.CipherCore.doFinal(CipherCore.java:824) at com.sun.crypto.provider.AESCipher.engineDoFinal(AESCipher.java:436)...

Sending Image From Android to C# webservice

c#,android,web-services,encoding,android-image
I'm sending image from Android to C# webservice. I always get a black image.Can anyone help? Android side: public void save(View v) { mBitmap = Bitmap.createBitmap(v.getWidth(), v.getHeight(), Bitmap.Config.RGB_565); ByteArrayOutputStream outputStream = new ByteArrayOutputStream(); mBitmap.compress(Bitmap.CompressFormat.JPEG,40, outputStream); byte[] imgByte = outputStream.toByteArray(); String base64Str = Base64.encodeToString(imgByte, Base64.DEFAULT); // Send base64Str to server }...

Chinese characters in properties files are not properly decoded [duplicate]

java,spring,jsp,encoding
This question already has an answer here: Internationalization using resource bundle properties in JSP, non-Latin text becomes Mojibake 1 answer I have used localisation for my application in Chinese and English. It is a Spring MVC application and I am getting the values from a properties file. I have...

Remove all characters which can't be decode in Python

python,encoding
I try to parse a html file with a Python script using the xml.etree.ElementTree module. The charset should be UTF-8 according to the header. But there is a strange character in the file. Therefore, the parser can't parse it. I opened the file in Notepad++ to see the character ....

Avoid manually decoding many strings in code

python,string,encoding,utf-8,refactoring
I work in Python. Since I have to put .decode('utf-8') very often, I would like to get it decoded automatically. Is there a way how to make all strings decoded from utf-8 automatically? And second question: Is there any disadvantage? PS: I have to decode especially strings where diacritics appears...

KeyError Pandas Dataframe (encoding index)

python-2.7,indexing,pandas,encoding,keyerror
I'm running the code below. It creates a couple of dataframes that takes a column in another dataframe that has a list of Conference Names, as its index. df_conf = pd.read_sql("select distinct Conference from publications where year>=1991 and length(conference)>1 order by conference", db) for index, row in df_conf.iterrows(): row[0]=row[0].encode("utf-8") df2=...

How to find UTF-8 reference of a composite unicode character

unicode,encoding,utf-8,character-encoding
At work, i have this issue where i need to find the UTF-8 reference of a composite unicode character. The character in question is a "n" with a "^" on top : n̂. This is represented in unicode by the character "n" (U+006E) followed by the circumflex accent (U+0302). What...

How to make sure a XDocument is saved with utf-8 file encoding?

c#,xml,unicode,encoding,utf-8
I am creating a Xml file with the following code (the byte array returned by Serialize() is written to a FileStream later): public byte[] Serialize() { using (var stream = new MemoryStream()) { WriteXmlToStream(stream); stream.Position = 0; using (var reader = new StreamReader(stream)) { string resultString = reader.ReadToEnd(); return Encoding.UTF8.GetBytes(resultString);...

XSS prevention and .innerHTML

javascript,encoding,xss,innerhtml
When I allow users to insert data as an argument to the JS innerHTML function like this: element.innerHTML = “User provided variable”; I understood that in order to prevent XSS, I have to HTML encode, and then JS encode the user input because the user could insert something like this:...

Perl internal representation of unicode string

perl,unicode,encoding,mojolicious
I am working on a perl + Mojolicious web application and my front-end send a POST query containing accents in a "a" parameter ("été") using charset utf-8 as I can spy in chrome network tab. But server side script decode that parameter using a charset that I didn't expect. I...

Force UTF-8 encoding in inline CSS

css,encoding,utf-8
I have inline CSS loading within an element which renders a close (X) icon on a popup overlay. When rendered on a page which is not in UTF-8 it is rendered badly with local characters instead. Code is as follows: .close{ position:absolute; top:-14px; right:-13px; cursor:pointer; color: #fff; border: 1px solid...

Python: difficulty converting ascii to unicode

python,unicode,encoding,utf-8
My goal: get the page source from a url and count all instances of a keyword within that page source How I am doing it: getting the pagesource via urllib2, looping through each char of the page source and comparing it to the keyword My problem: my keyword is encoded...

PHP / MySQL: Certain characters not being encoded properly and appearing as question marks

php,mysql,encoding,utf-8,character-encoding
I am new to PHP and MySQL and hope someone can help me with this. I have a MySQL db with a table called "myTable". Both the server connection collation and the single columns containing text are set up with the data type "utf8_general_ci" and all characters appear correctly within...

How to display special chars in html [duplicate]

php,html5,encoding,character-encoding
This question already has an answer here: UTF-8 all the way through 14 answers I have a web page and suddenly it is displaying spcecial characters as asian characters like ñ is display as 帽 this is the head of my template <!DOCTYPE html> <html lang="es"> <head> <meta charset="utf-8">...

How to encode special characters in e-mail-adresses

perl,email,encoding,special-characters
E-Mail-Adresses do not only consist of this parts: [email protected] The complete string in the next line (including the part between the quotation marks, the quotation marks itself and the angle brackets) is also a valid address: "John Doe" <[email protected]> And when I replace "John Doe" with my own name, I...

UnicodeDecodeError for medieval characters

python,unicode,encoding,utf-8,nltk
I am trying to run an nltk tokenize program on medieval texts. These texts use medieval characters such as yogh (ȝ), thorn (þ), and eth (ð). When I run the program (pasted below) with standard unicode (utf-8) encoding, I get the following error: Traceback (most recent call last): File "me_scraper_redux2.py",...

Does HTML Encoding have any cons?

asp.net-mvc,razor,encoding,utf-8,xss
I develop a project on ASP.NET MVC framework. All files and charsets are in UTF-8. I'm using model bindings and in some of my models the display property includes some accented chars or single/double quotes. As Razor engine automatically encodes helpers (ie. DisplayNameFor) the accented chars and quotes are encoded....

C++: socket encoding (working with TeamSpeak)

c++,encoding,listbox
As I'm currently working on a program for a TeamSpeak server, I need to retrieve the names of the currently online users which I'm doing with sockets - that's working fine so far.In my UI I'm displaying all clients in a ListBox which is basically working. Nevertheless I'm having problems...

How to work arround php notice raised incorrecty when posting a string that begins with an @?

php,post,curl,encoding
I want to perform a post request using PHP and Curl. This is how I set my post parameters: curl_setopt($ch,CURLOPT_POSTFIELDS,array('name'=> $name)); When the String in $name happens to contain an @ character at the beginning I get the following error. [8192] curl_setopt(): The usage of the @filename API for file...

debugging ByteArrayOutputStream string comparison

java,encoding
I have a strange problem with Java's ByteArrayOutputStream.. public static void main(String[] args) { ByteArrayOutputStream baos = new ByteArrayOutputStream(); PrintStream ps = new PrintStream(baos); String expected = "{x={x=4, y=true}, y=true}"; printToStream(ps); try { String content = new String(baos.toByteArray()); System.out.println(">content : "+content); System.out.println("-expected: "+expected); printBytes(content); printBytes(expected); if(content.equals(expected)==false) //HERE throw new RuntimeException(...

Encoding warning in Maven Selenium WebDriver project executing in IntelliJ IDEA [duplicate]

java,maven,encoding,selenium-webdriver
This question already has an answer here: How to configure encoding in maven 2 answers I have a Maven Selenium WebDriver project in IntelliJ IDEA 14.0.2. My pom.xml file is as below: <?xml version="1.0" encoding="UTF-8"?> <project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd"> <modelVersion>4.0.0</modelVersion>...

What is the right CSV encoding for c# to open with Excel?

c#,excel,csv,encoding,streamwriter
I write a CSV with my C# CSVWriter based on a StreamWriter. In my data I have a lot of special characters like "Bávaro". So when I use UTF-8 or ASCII to encode my CSV I can't get the "á" but I can open it in Excel afterwards perfectly. When...

HttpUtility.UrlDecode encoding from UTF-8 to ASCII

c#,encoding
answers2 = HttpUtility.UrlDecode(Model.FindOne<UserResult>(ur => ur.UserID == user.UserID && ur.ModuleID == 2 && ur.TopicID == 9 && ur.ActivityID == 2 && ur.QuestionID == 2).Results, Encoding.ASCII).Split('|'); The result field contains German characters encoded as ASCII in the SQL database. I'm using a bulk SMS sending service which requires that special characters...

Encoding error in Rails 4 when querying mysql DB

mysql,ruby,ruby-on-rails-4,encoding,utf-8
I am receiving this error under certain situations when querying my db which is on an aws server (mysql). I have the db configured to utf8 and have the rails app set up to utf-8 as well in the config/application.rb file. Any suggestions? AwsCourse Load (36.5ms) SELECT COURSE.* FROM COURSE...

Extracting Double Byte Characters/substring from a UTF-8 formatted String

java,string,encoding,utf-8
I'm trying to extract emojis and other special Characters from Strings for further processing (e.g. a String contains '😅' as one of its Characters). But neither string.charAt(i) nor string.substring(i, i+1) work for me. The original String is formatted in UTF-8 and this means, that the escaped form of the above...

JavaMail MimeMessage.getContent unsupported encoding

email,encoding,javamail
I´m having a problem with parsing some type of mail encoding with charset=t.101-g2 I´m using JavaMail in Grails directly using the lib, getting the unread emails from IMAP, and for each message try to get the content with Object msgContent = msg.getContent() And that line is throwing the error. The...

Why are some of my HTML internal links working and others not?

html,encoding,hyperlink,anchor,href
Help, I am totally new to this world. I am making a menu bar on front page to with links to other pages on the site, here is example of code: <ul id="menu"> <li><a href="file:///Users/tamborine/Downloads/about_page.htm">About</a></li> <li><a href=“file:///Users/tamborine/Downloads/the_therapist_page.html”>The Therapist</a></li> <li><a href=“file:///Users/tamborine/Downloads/pay_what_you_can_page.htm”>Pay what you can</a></li> <li><a...

Encode publicKey on Java Card

java,encoding,javacard,ecdsa,pkcs#8
How to encode an ECDSA PublicKey on Java Card so that after I can decode it on another platform (e.g. sending the encoded key in a response APDU and processing it in a standard Java application)? keyPair.getPublic().getEncoded() on Java would do the trick with PKCS#8 encoding, but as far as...

which unicode should be used UTF-8 and UTF-16 when deal with accented characters in java response to Ajax

java,ajax,unicode,encoding,utf-8
First of all, what is main difference between UTF-8 and UTF-16, and when dealing with accented character in ajax to java, I facing an issue which is in this question [java response to ajax with accented characters - Garbled many thanks if you can help me with this....

URL-encoding VARCHAR and TEXT values in PostgreSQL

postgresql,encoding,plpgsql,url-encoding
Are there any well-known PL functions/libraries for extending a PostgreSQL (9.4.1) database with URL encoding (also known as percent encoding) capabilities? Here's an example of the intended functionality: Input string: International donor day: give blood for a good cause! Output string: International%20donor%20day%3A%20give%20blood%20for%20a%20good%20cause%21 I guess an alternative would be to roll...

Need method to convert Excel to Tab Delimited txt file in Unicode encoding

excel,unicode,encoding
I have some excel files containing some French characters, for example "Jean-Léon Gérôme" when I convert the excel file to tab delimited txt file, the default encoding is ANSI and the French characters became Gibberish, so I need a method that can convert the excel file to tab delimited file...

What encoding does [BouncyCastle] PKCS10CertificationRequest.getEncoded() return?

java,encoding,cryptography,bouncycastle
Does it return DER encoded data, or some other format? The Javadoc I've been able to find leaves something to be desired details-wise......

R- Changing encoding of column in dataframe?

r,encoding,data.frame
I am trying to change the encoding of a column in a dataframe. stri_enc_mark(data_updated$text) # [1] "UTF-8" "ASCII" "ASCII" "UTF-8" "ASCII" "ASCII" "UTF-8" "UTF-8" "UTF-8" # [10] "ASCII" "ASCII" "UTF-8" "ASCII" "UTF-8" "ASCII" "UTF-8" "ASCII" "UTF-8" # [19] "ASCII" "UTF-8" "ASCII" "UTF-8" "ASCII" "UTF-8" "UTF-8" "ASCII" "ASCII" # [28] "ASCII"...

Convert unicode URL to ASCII

php,unicode,encoding,ascii
I'm writing a PHP application that accepts an URL from the user, and then processes it with by making some calls to binaries with system()*. However, to avoid many complications that arise with this, I'm trying to convert the URL, which may contain Unicode characters, into ASCII characters. Let's say...

PHP: PNG readfile encoding issue

php,encoding,png,php-gd
I have this code which outputs a PNG file to the browser header('Content-type: image/png'); header("Content-Length: " . filesize($cache_file)); readfile($cache_file); exit(); The file $cache_file was saved like this imagepng(self::$image, self::$cache); The browser is displaying a broken image. When I download it and open it with npp, I find that it's encoded...

Encoding and decoding strings with the use of java.util.Base64

java,encoding,base64,java-8,decoding
I try to find a way to encode and decode strings with the use of java.util.Base64.Encoder and Decoder classes. Unfortunately, it is impossible to call encode and decode methods statically, so I've created references to Encoder and Decoder classes. But in order to create an instance object for each of...

Detecting corrupt characters in UTF-8 encoded text file

regex,encoding,awk,utf-8,scripting
I have a text file that was edited with the wrong character encoding and thus has some mojibake and corrupt characters in some of the strings when I open it using UTF-8. What scripting language would be the most efficient at detecting these corrupt characters? Perl is not an option....

Can't reproduce working C bitwise encoding function in Python

python,security,python-2.7,encoding,reverse-engineering
I'm reverse engineering a proprietary network protocol that generates a (static) one-time pad on launch and then uses that to encode/decode each packet it sends/receives. It uses the one-time pad in a series of complex XORs, shifts, and multiplications. I have produced the following C code after walking through the...

Who is adding “\n” in Base64 encoded image when I write it in a file? Java

java,android,image,encoding,base64
WHAT I'M DOING I need to send via HTTPS request a JsonArray with some data and images in Base64 encoded strings. This works well if data is stored in memory. Now, I need to avoid load all data in memory and I'm creating a temporally file in android device with...

char encoding doing mysql insert with callablestatement or jdbctemplate

java,mysql,encoding,jdbctemplate,callable-statement
I am having issues with char encoding when performing an insert into a mysql table. The table field I am focused on have the following properties: Collation:utf8_general_ci varchar(255) The problem occurs with chinese and foreign characters. I tried using a callablestatement to call a stored procedure but it does not...

ODBC php Excel Encoding Issues

php,excel,encoding,utf-8,odbc
I'm making an ODBC connection with php to an Excel file. However when I encounter characters like "~", "^", "´" they appear messed up as this: Mês -> M?s Formação -> Forma??o I'm doing the following utf-8 treatment just to get the column names: $con = odbc_connect($odbcName, $dbUser, $dbPassword) or...

How do you use unicode in Qt? [duplicate]

c++,qt,unicode,encoding,qt4
This question already has an answer here: Retrieve Unicode code points > U+FFFF from QChar 2 answers I would like to use the magnifying glass (U+1F50E) Unicode symbol in a QLineEdit field. I figured out how to use 16-bit unicode symbols using QChar however I have no idea how...

Django UnicodeDecodeError in model object

django,encoding
this is another UnicodeDecodeError I'm dealing with in django. I can't find the way to solve it. I'm trying to create an object: nivel_obj = Nivel.objects.filter(id=nivel_id) nueva_matricula = Matricula(nivel=nivel_obj, ano_lectivo=ano_lectivo, alumno=a) nueva_matricula.save() The "Matricula" object has a "nivel_obj" that is a Foreign Key. The "nivel_obj" has a name that is...

MongoDB BSON codec not being used while encoding object

java,mongodb,encoding,bson
I'm attempting to store an object in a MongoDB database (using MongoDB 3.0.2) and am getting a CodecConfigurationException when attempting to encode the object with error message "Can't find a codec for class java.time.LocalDate". I have written and included a codec for the LocalDate objects. Details follow. The object, DutyBlock,...

Converting Unicode codepoints to UTF-8 in C using iconv

c,unicode,encoding,utf-8,iconv
I want to convert a 32-bit value, which represents a Unicode codepoint, into a sequence of chars which is the utf-8 encoded string containing only the character corresponding to the codepoint. For example, I want to turn the value 955 into the utf-8 encoded string "λ". I tried to do...

Problems with filename encoding on Linux when created through Java

java,linux,encoding
I'm having a problem, and I've been stuck with it for quite some time now. The problem is, I can't get file names to support special characters (only file names, file content is ok). Instead of them, file name is created with question marks. Now, from what I know (and...

How to set the filename encoding globally for a python interpreter?

python,encoding,utf-8,filenames
In a project, we need to open many files all over the place in the source code. We know that all filenames on disk are encoded in utf-8 and all filenames are proceed as Unicode internally. Is there a way to set the filename encoding globally for the running interpreter,...

ASP MVC Post Encoding Shift-Jis

asp.net-mvc,encoding,shift-jis
I have a form that is sits in a plain .html file with the following meta tag: <meta http-equiv="Content-Type" content="text/html; charset=shift_jis"> The service I'm posting a form to requires that the data is in shift-jis encoding. When I post this form by just opening this .html file with Chrome, then...