Scrapinghub/Splash website page fetching time increasing exponentially with parellel thread - webkit

In my trial, I hit splash instance with 50 parallel threads. Each thread will get the page source of the URL. My splash instance default slots value is 50. Here, website fetching time increases exponentially with the number of parallel threads. I can get the perfect HTML source for 50 URLs. But the time increases from 2 seconds to 45 seconds from 1st URL to 50th URL respectively. Please help me to reduce the time for fetching the page source.
My sample java code is
public class SplashThread implements Runnable {
private static final String splash ="http://localhost:8050/execute";
private String script = URLEncoder.encode("function main(splash, args) \n" +
"splash:go(args.url)\n"+
" splash.images_enabled = false\n" +
" splash:on_request(function(request)\n" +
" if string.find(request.url, \".css\")~= nil then\n" +
" request.abort()\n" +
" end\n" +
"end)\n" +
"local html = splash:html()\n" +
"return html\n"+
"end","UTF-8");
private String url =null;
public SplashThread(String url) throws UnsupportedEncodingException {
this.url = url;
}
#Override
public void run() {
HttpClientUtil clientUtil =null;
JSONObject json =null;
try {
Properties queryParms = new Properties();
queryParms.put("url",url);
queryParms.put("timeout","85.0");
queryParms.put("lua_source",script);
clientUtil = new HttpClientUtil(-1,false);
HttpResponse response = clientUtil.doGet(splash,queryParms,null);
String resp = ScrapyUtil.getResponseString(response,"UTF-8");
}
catch (Exception e){
e.printStackTrace();
System.out.println("JSN}ON :: "+json);
}
finally {
if(clientUtil!=null){
try {
clientUtil.close();
} catch (IOException e) {
e.printStackTrace();
}
}
}
}
}
I am scheduling 50 threads of this runnable object with ScheduledExecutorService.
If I am fecthing the page source once by one, It will working perfectly. But I need to fecth concurrently.

Related

WebFlux issue when creating a global publisher subscription

I have written to the publisher and it works if they subscribe to it every time. The problem arises when I try to make it global so that all subscribers receive the same messages.
Here is my code :
public class MyPublisherDemo {
private final ArrayBlockingQueue<SavedEvent> myBlockingQueue = new ArrayBlockingQueue<>(100, true);
private final Flux<SavedEvent> fx = Flux.create(event -> {
try{
while(true){
Thread.sleep(5000);
if(myBlockingQueue.isEmpty()){
System.out.println("back");
break;
}
event.next(myBlockingQueue.poll());
System.out.println("next");
}
event.complete();
System.out.println("complete");
}catch (Exception e) {
event.error(new RuntimeException(e));
}
});
private final ConnectableFlux<SavedEvent> cf = fx.publish();
public Flux<String> getFluxTest(SavedEvent savedEvent){
myBlockingQueue.add(savedEvent);
cf.connect();
return cf.map(SavedEvent::getOccurredEvent);
}
public int getQueueSize(){
return myBlockingQueue.size();
}
}
And I wrote the following test to check:
#Test
public void test5(){
try {
MyPublisherDemo myPublisherDemo = new MyPublisherDemo();
Runnable task = () -> {
System.out.println(Thread.currentThread().getName() + " Run");
myPublisherDemo.getFluxTest(new SavedEvent().builder()
.id(1L)
.occurredEvent("(" + Thread.currentThread().getName() + ")" + " Coffee Machine start")
.eventTime(LocalDateTime.now())
.fillTheWaterTank(1000)
.fillCoffeeTank(1000)
.build())
.subscribe(
event -> System.out.println(Thread.currentThread().getName() + " " + event),
System.out::println,
() -> System.out.println(Thread.currentThread().getName() + " end")
);
};
Thread th1 = new Thread(task, "thread-1");
th1.start();
Thread.sleep(2000);
Thread th2 = new Thread(task, "thread-2");
th2.start();
Thread.sleep(20000);
System.out.println("Queue size = " + myPublisherDemo.getQueueSize());
System.out.println("Basic thread run. End program.");
}catch (InterruptedException e) {
throw new RuntimeException(e);
}
}
The structure of SavedEvent I think is clear from builder().
The point of the publisher is to wait for the event for 5 seconds if the current event is still being processed.
The version I used before:
private final ArrayBlockingQueue<SavedEvent> myBlockingQueue = new ArrayBlockingQueue<>(100, true);
private final AtomicInteger counterRequest = new AtomicInteger(100);
private final Flux<SavedEvent> fx = Flux.fromIterable(myBlockingQueue)
.delayElements(Duration.ofSeconds(7))
.map(data ->{
myBlockingQueue.poll();
counterRequest.decrementAndGet();
return data;
})
.repeat(counterRequest.get());
private final ConnectableFlux<SavedEvent> cf = fx.publish();
Problem when working with two subscribers they don't receive all messages.
thread-1 Run
thread-2 Run
thread-1 (thread-1) Coffee Machine start
next
thread-1 (thread-2) Coffee Machine start
next
back
thread-1 end
complete
Queue size = 0
Basic thread run. End program.
The result I'm expecting:
thread-1 Run
thread-2 Run
thread-1 (thread-1) Coffee Machine start
thread-2 (thread-1) Coffee Machine start
next
thread-2 (thread-2) Coffee Machine start
thread-1 (thread-2) Coffee Machine start
next
back
thread-1 end
complete
back
thread-2 end
complete
Queue size = 0
Basic thread run. End program.
If the error is in how I'm trying to implement the publisher, please write how it should be done correctly. Thank you in advance.
While experimenting with RxJava I found a solution on the net:
let myObservable = Observable.just(1).publish()
print("Subscribing")
myObservable.subscribe(onNext: {
print("first = \($0)")
})
myObservable.subscribe(onNext: {
print("second = \($0)")
})
DispatchQueue.main.asyncAfter(deadline: .now() + 3) {
print("Calling connect after 3 seconds")
myObservable.connect()
}
It's not obvious, but for the first subscriber, you first need to have a subscription to the publisher and then you need to start the publisher.I did not understand why this is so. It works.
public class MyPublisherDemo {
private final ArrayBlockingQueue<SavedEvent> myBlockingQueue = new ArrayBlockingQueue<>(100, true);
private final AtomicBoolean connectFlag = new AtomicBoolean(false);
private final ConnectableFlux<String> myEventGenerator = Flux.create(event ->{
try{
while(true){
Thread.sleep(5000);
if(myBlockingQueue.isEmpty()){
break;
}
event.next(myBlockingQueue.poll());
}
connectFlag.getAndSet(false);
event.complete();
}catch (Exception e) {
event.error(new RuntimeException(e));
}
}).map(event -> {
SavedEvent se = (SavedEvent) event;
return se.getOccurredEvent();
}).publish();
public Flux<String> getFluxTest(SavedEvent savedEvent){
myBlockingQueue.add(savedEvent);
delayedStart();
return myEventGenerator;
}
public int getQueueSize(){
return myBlockingQueue.size();
}
private void delayedStart(){
if (!connectFlag.getAndSet(true)) {
Runnable task = () -> {
try {
Thread.sleep(500);
myEventGenerator.connect();
} catch (Exception e) {
System.err.println(e);
}
};
Thread th1 = new Thread(task);
th1.start();
}
}
}

How to test login page with multiple webpage(present in notepad) using selenium web driver Java?

I am testing like 100 url everything is working good. my problem is when the test starts it get web url from notepad then open the url here my process is to check the login or not in some case if the for example 10th url shows 404 error the test terminate here it cant process further urls. Any solution for this process...
my codes
public class File {
#Test(dataProvider="testdata")
public void sum(String url)
{
System.out.println(url);
}
#DataProvider
public Object[][] testdata(){
int count=20;
Object[][] obj = new Object[count][1];
for(int i=0;i<=count;i++)
{
String fileName = "E:\\ework\\Web\\bin\\Websearch\\test.txt";
try {
String line = null;
FileReader fileReader = new FileReader(fileName);
BufferedReader bufferedReader = new BufferedReader(fileReader);
while((line = bufferedReader.readLine()) != null)
{
String[] abs={line};
int size = abs.length;
for(int j=0;j<=size;j++)
{
obj[i][0]=abs[j];
}
}
bufferedReader.close();
}
catch(FileNotFoundException ex) {
System.out.println(
"Unable to open file '" +
fileName + "'");
}
catch(IOException ex) {
System.out.println(
"Error reading file '"
+ fileName + "'");
}
`}
return obj;
}
}
thank you.:)
If you are using framework like TestNG then look at DataProviders for the same.
Use Try{} Catch(Exception e){} block. Insert your assertion in try block and once the assertion fails then try to capture the exception in some list or other data structure and after all urls are processed then iterate over the exception list and report it in your Reports or logs.

Using LocationListener (with google play service) as service consume too much battery (Android 4.4.2)

I have a service that returns the current position every 30 seconds, my problem is that the service consumes about 40% of the battery. The GPS icon is always active, even if I increase the time interval. I see this problem with my Nexus 4 android 4.4.2. Once the callback OnLocationChanged is called, the GPS is awake all the time and this consumes the entire battery. With my other Phone Nexus One android 2.2.3 , I do not see this problem, the GPS turns on every 30 seconds and turns off. And I have no battery consommation. The service use Location Request, with .setPriority(LocationRequest.PRIORITY_HIGH_ACCURACY), excepted to use GPS and Network and is not the case. I thinks there is a problem with android 4.4.2 or is Google play service
Here my service code:
public class MyServiceGpsDebug extends Service implements
GooglePlayServicesClient.ConnectionCallbacks,
GooglePlayServicesClient.OnConnectionFailedListener,LocationListener
{
IBinder mBinder = new LocalBinder();
private LocationClient mLocationClient;
private LocationRequest mLocationRequest;
// Flag that indicates if a request is underway.
private boolean mInProgress;
private Boolean servicesAvailable = false;
public class LocalBinder extends Binder
{
public MyServiceGpsDebug getServerInstance()
{
return MyServiceGpsDebug.this;
}
}
#Override
public void onCreate()
{
super.onCreate();
mInProgress = false;
// Create the LocationRequest object
mLocationRequest = LocationRequest.create();
// Use high accuracy
mLocationRequest.setPriority(LocationRequest.PRIORITY_HIGH_ACCURACY);
// Set the update interval to 5 seconds
mLocationRequest.setInterval(Constants.UPDATE_INTERVAL);
// Set the fastest update interval to 1 second
mLocationRequest.setFastestInterval(Constants.FASTEST_INTERVAL);
servicesAvailable = servicesConnected();
mLocationClient = new LocationClient(this, this, this);
}
private boolean servicesConnected()
{
// Check that Google Play services is available
int resultCode = GooglePlayServicesUtil.isGooglePlayServicesAvailable(this);
// If Google Play services is available
if (ConnectionResult.SUCCESS == resultCode)
{
return true;
}
else
{
return false;
}
}
public int onStartCommand (Intent intent, int flags, int startId)
{
super.onStartCommand(intent, flags, startId);
if(!servicesAvailable || mLocationClient.isConnected() || mInProgress)
return START_STICKY;
setUpLocationClientIfNeeded();
if(!mLocationClient.isConnected() || !mLocationClient.isConnecting() && !mInProgress)
{
appendLog(DateFormat.getDateTimeInstance().format(new Date()) + ": Started", Constants.LOG_FILE);
mInProgress = true;
mLocationClient.connect();
}
return START_STICKY;
}
private void setUpLocationClientIfNeeded()
{
if(mLocationClient == null)
mLocationClient = new LocationClient(this, this, this);
}
#Override
public void onLocationChanged(Location location)
{
// Report to the UI that the location was updated
String msg = Double.toString(location.getLatitude()) + "," + Double.toString(location.getLongitude());
Log.d("debug", msg);
// Toast.makeText(this, msg, Toast.LENGTH_SHORT).show();
appendLog(msg, Constants.LOCATION_FILE);
}
#Override
public IBinder onBind(Intent intent)
{
return mBinder;
}
public String getTime()
{
SimpleDateFormat mDateFormat = new SimpleDateFormat("yyyy-MM-dd HH:mm:ss");
return mDateFormat.format(new Date());
}
public void appendLog(String text, String filename)
{
File logFile = new File(filename);
if (!logFile.exists())
{
try
{
logFile.createNewFile();
}
catch (IOException e)
{
// TODO Auto-generated catch block
e.printStackTrace();
}
}
try
{
//BufferedWriter for performance, true to set append to file flag
BufferedWriter buf = new BufferedWriter(new FileWriter(logFile, true));
buf.append(text);
buf.newLine();
buf.close();
}
catch (IOException e)
{
// TODO Auto-generated catch block
e.printStackTrace();
}
}
#Override
public void onDestroy()
{
// Turn off the request flag
mInProgress = false;
if(servicesAvailable && mLocationClient != null)
{
mLocationClient.removeLocationUpdates(this);
// Destroy the current location client
mLocationClient = null;
}
// Display the connection status
// Toast.makeText(this, DateFormat.getDateTimeInstance().format(new Date()) + ": Disconnected. Please re-connect.", Toast.LENGTH_SHORT).show();
appendLog(DateFormat.getDateTimeInstance().format(new Date()) + ": Stopped", Constants.LOG_FILE);
super.onDestroy();
}
#Override
public void onConnected(Bundle bundle)
{
// Request location updates using static settings
mLocationClient.requestLocationUpdates(mLocationRequest, this);
appendLog(DateFormat.getDateTimeInstance().format(new Date()) + ": Connected", Constants.LOG_FILE);
}
#Override
public void onDisconnected()
{
// Turn off the request flag
mInProgress = false;
// Destroy the current location client
mLocationClient = null;
// Display the connection status
// Toast.makeText(this, DateFormat.getDateTimeInstance().format(new Date()) + ": Disconnected. Please re-connect.", Toast.LENGTH_SHORT).show();
appendLog(DateFormat.getDateTimeInstance().format(new Date()) + ": Disconnected", Constants.LOG_FILE);
}
#Override
public void onConnectionFailed(ConnectionResult connectionResult)
{
mInProgress = false;
if (connectionResult.hasResolution())
{
// If no resolution is available, display an error dialog
} else {
}
}
public final class Constants
{
// Milliseconds per second
private static final int MILLISECONDS_PER_SECOND = 1000;
// Update frequency in seconds
private static final int UPDATE_INTERVAL_IN_SECONDS = 60;
// Update frequency in milliseconds
public static final long UPDATE_INTERVAL = MILLISECONDS_PER_SECOND * UPDATE_INTERVAL_IN_SECONDS;
// The fastest update frequency, in seconds
private static final int FASTEST_INTERVAL_IN_SECONDS = 60;
// A fast frequency ceiling in milliseconds
public static final long FASTEST_INTERVAL = MILLISECONDS_PER_SECOND * FASTEST_INTERVAL_IN_SECONDS;
// Stores the lat / long pairs in a text file
public static final String LOCATION_FILE = "sdcard/location.txt";
// Stores the connect / disconnect data in a text file
public static final String LOG_FILE = "sdcard/log.txt";
/**
* Suppress default constructor for noninstantiability
*/
private Constants() {
throw new AssertionError();
}
}
}
I have similar problem with Nexus 4. My app has a service which uses location updates (fusion location or android providers, not both). Everything works OK for all android phones but in Nexus4 after some time the phone gets hot and slow even if i kill the app (through DDMS, 100% stopped). The only solution is to kill Google play services .
I think that there is a bug in play services for nexus 4. There is a dirty solution to kill Google Play Services every 30mins for example if phone=nexus4, but i do not know if it is possible

android media player connecting status with audio streaming link(ie 2%,4%...100% then online radio start to play) in android

how i get buffering status while media player trying to connect audio streaming link(ie 2%,4%...100% then online radio start to play) in android.
this is my code.
but i have no idea how i solve my problem.thanks is advance to any kind of help.
player.setOnBufferingUpdateListener(new OnBufferingUpdateListener() {
public void onBufferingUpdate(MediaPlayer mp, int percent) {
playSeekBar.setSecondaryProgress(percent);
Log.i("Buffering", "" + percent);
}
});
i solve this problem. here is the link. http://coderfriend.blogspot.com/
as per request here i share blog content..
when user click play button to play radio then i want to show connecting status(buffering 1%,2%.. 99%). when status will be 100% radio start to play. i was face problem to solve this. so here i share this code for all.
//at first create this class
public class StreamingMediaPlayer {
private static final int INTIAL_KB_BUFFER = 96*10/8;//assume 96kbps*10secs/8bits per byte
private TextView textStreamed;
private ImageButton playButton;
private ProgressBar progressBar;
// Track for display by progressBar
private long mediaLengthInKb, mediaLengthInSeconds;
private int totalKbRead = 0;
// Create Handler to call View updates on the main UI thread.
private final Handler handler = new Handler();
private MediaPlayer mediaPlayer;
private File downloadingMediaFile;
private boolean isInterrupted;
private Context context;
private int counter = 0;
public StreamingMediaPlayer(Context context,TextView textStreamed, ImageButton playButton, Button streamButton,ProgressBar progressBar)
{
this.context = context;
this.textStreamed = textStreamed;
this.playButton = playButton;
this.progressBar = progressBar;
}
/**
* Progressivly download the media to a temporary location and update the MediaPlayer as new content becomes available.
*/
public void startStreaming(final String mediaUrl, long mediaLengthInKb, long mediaLengthInSeconds) throws IOException {
this.mediaLengthInKb = mediaLengthInKb;
this.mediaLengthInSeconds = mediaLengthInSeconds;
Runnable r = new Runnable() {
public void run() {
try {
downloadAudioIncrement(mediaUrl);
} catch (IOException e) {
Log.e(getClass().getName(), "Unable to initialize the MediaPlayer for fileUrl=" + mediaUrl, e);
return;
}
}
};
new Thread(r).start();
}
/**
* Download the url stream to a temporary location and then call the setDataSource
* for that local file
*/
public void downloadAudioIncrement(String mediaUrl) throws IOException {
URLConnection cn = new URL(mediaUrl).openConnection();
cn.connect();
InputStream stream = cn.getInputStream();
if (stream == null) {
Log.e(getClass().getName(), "Unable to create InputStream for mediaUrl:" + mediaUrl);
}
downloadingMediaFile = new File(context.getCacheDir(),"downloadingMedia.dat");
if (downloadingMediaFile.exists()) {
downloadingMediaFile.delete();
}
FileOutputStream out = new FileOutputStream(downloadingMediaFile);
byte buf[] = new byte[16384];
int totalBytesRead = 0, incrementalBytesRead = 0;
do {
int numread = stream.read(buf);
if (numread <= 0)
break;
out.write(buf, 0, numread);
totalBytesRead += numread;
incrementalBytesRead += numread;
totalKbRead = totalBytesRead/1000;
testMediaBuffer();
fireDataLoadUpdate();
} while (validateNotInterrupted());
stream.close();
if (validateNotInterrupted()) {
fireDataFullyLoaded();
}
}
private boolean validateNotInterrupted() {
if (isInterrupted) {
if (mediaPlayer != null) {
mediaPlayer.pause();
//mediaPlayer.release();
}
return false;
} else {
return true;
}
}
/**
* Test whether we need to transfer buffered data to the MediaPlayer.
* Interacting with MediaPlayer on non-main UI thread can causes crashes to so perform this using a Handler.
*/
private void testMediaBuffer() {
Runnable updater = new Runnable() {
public void run() {
if (mediaPlayer == null) {
// Only create the MediaPlayer once we have the minimum buffered data
if ( totalKbRead >= INTIAL_KB_BUFFER) {
try {
startMediaPlayer();
} catch (Exception e) {
Log.e(getClass().getName(), "Error copying buffered conent.", e);
}
}
} else if ( mediaPlayer.getDuration() - mediaPlayer.getCurrentPosition() <= 1000 ){
// NOTE: The media player has stopped at the end so transfer any existing buffered data
// We test for < 1second of data because the media player can stop when there is still
// a few milliseconds of data left to play
transferBufferToMediaPlayer();
}
}
};
handler.post(updater);
}
private void startMediaPlayer() {
try {
File bufferedFile = new File(context.getCacheDir(),"playingMedia" + (counter++) + ".dat");
// We double buffer the data to avoid potential read/write errors that could happen if the
// download thread attempted to write at the same time the MediaPlayer was trying to read.
// For example, we can't guarantee that the MediaPlayer won't open a file for playing and leave it locked while
// the media is playing. This would permanently deadlock the file download. To avoid such a deadloack,
// we move the currently loaded data to a temporary buffer file that we start playing while the remaining
// data downloads.
moveFile(downloadingMediaFile,bufferedFile);
Log.e(getClass().getName(),"Buffered File path: " + bufferedFile.getAbsolutePath());
Log.e(getClass().getName(),"Buffered File length: " + bufferedFile.length()+"");
mediaPlayer = createMediaPlayer(bufferedFile);
// We have pre-loaded enough content and started the MediaPlayer so update the buttons & progress meters.
mediaPlayer.start();
startPlayProgressUpdater();
playButton.setEnabled(true);
} catch (IOException e) {
Log.e(getClass().getName(), "Error initializing the MediaPlayer.", e);
return;
}
}
private MediaPlayer createMediaPlayer(File mediaFile)
throws IOException {
MediaPlayer mPlayer = new MediaPlayer();
mPlayer.setOnErrorListener(
new MediaPlayer.OnErrorListener() {
public boolean onError(MediaPlayer mp, int what, int extra) {
Log.e(getClass().getName(), "Error in MediaPlayer: (" + what +") with extra (" +extra +")" );
return false;
}
});
// It appears that for security/permission reasons, it is better to pass a FileDescriptor rather than a direct path to the File.
// Also I have seen errors such as "PVMFErrNotSupported" and "Prepare failed.: status=0x1" if a file path String is passed to
// setDataSource(). So unless otherwise noted, we use a FileDescriptor here.
FileInputStream fis = new FileInputStream(mediaFile);
mPlayer.setDataSource(fis.getFD());
mPlayer.prepare();
return mPlayer;
}
/**
* Transfer buffered data to the MediaPlayer.
* NOTE: Interacting with a MediaPlayer on a non-main UI thread can cause thread-lock and crashes so
* this method should always be called using a Handler.
*/
private void transferBufferToMediaPlayer() {
try {
// First determine if we need to restart the player after transferring data...e.g. perhaps the user pressed pause
boolean wasPlaying = mediaPlayer.isPlaying();
int curPosition = mediaPlayer.getCurrentPosition();
// Copy the currently downloaded content to a new buffered File. Store the old File for deleting later.
File oldBufferedFile = new File(context.getCacheDir(),"playingMedia" + counter + ".dat");
File bufferedFile = new File(context.getCacheDir(),"playingMedia" + (counter++) + ".dat");
// This may be the last buffered File so ask that it be delete on exit. If it's already deleted, then this won't mean anything. If you want to
// keep and track fully downloaded files for later use, write caching code and please send me a copy.
bufferedFile.delete();
moveFile(downloadingMediaFile,bufferedFile);
// Pause the current player now as we are about to create and start a new one. So far (Android v1.5),
// this always happens so quickly that the user never realized we've stopped the player and started a new one
mediaPlayer.pause();
// Create a new MediaPlayer rather than try to re-prepare the prior one.
mediaPlayer = createMediaPlayer(bufferedFile);
mediaPlayer.seekTo(curPosition);
// Restart if at end of prior buffered content or mediaPlayer was previously playing.
// NOTE: We test for < 1second of data because the media player can stop when there is still
// a few milliseconds of data left to play
boolean atEndOfFile = mediaPlayer.getDuration() - mediaPlayer.getCurrentPosition() <= 1000;
if (wasPlaying || atEndOfFile){
mediaPlayer.start();
}
// Lastly delete the previously playing buffered File as it's no longer needed.
oldBufferedFile.delete();
}catch (Exception e) {
Log.e(getClass().getName(), "Error updating to newly loaded content.", e);
}
}
private void fireDataLoadUpdate() {
Runnable updater = new Runnable() {
public void run() {
if((totalKbRead>19)&&(totalKbRead<120))
textStreamed.setText((totalKbRead-19 + "% Buffering"));//show buffering status.. ie 1%,2%. in ui
else if(totalKbRead<19)
textStreamed.setText(("Connecting..."));
else
textStreamed.setText((""));
float loadProgress = ((float)totalKbRead/(float)mediaLengthInKb);
progressBar.setSecondaryProgress((int)(loadProgress*100));
}
};
handler.post(updater);
}
private void fireDataFullyLoaded() {
Runnable updater = new Runnable() {
public void run() {
transferBufferToMediaPlayer();
// Delete the downloaded File as it's now been transferred to the currently playing buffer file.
downloadingMediaFile.delete();
textStreamed.setText(("Audio full loaded: " + totalKbRead + " Kb read"));
}
};
handler.post(updater);
}
public MediaPlayer getMediaPlayer() {
return mediaPlayer;
}
public void startPlayProgressUpdater() {
float progress = (((float)mediaPlayer.getCurrentPosition()/1000)/mediaLengthInSeconds);
progressBar.setProgress((int)(progress*100));
if (mediaPlayer.isPlaying()) {
Runnable notification = new Runnable() {
public void run() {
startPlayProgressUpdater();
}
};
handler.postDelayed(notification,1000);
}
}
public void interrupt() {
playButton.setEnabled(false);
isInterrupted = true;
validateNotInterrupted();
}
/**
* Move the file in oldLocation to newLocation.
*/
public void moveFile(File oldLocation, File newLocation)
throws IOException {
if ( oldLocation.exists( )) {
BufferedInputStream reader = new BufferedInputStream( new FileInputStream(oldLocation) );
BufferedOutputStream writer = new BufferedOutputStream( new FileOutputStream(newLocation, false));
try {
byte[] buff = new byte[8192];
int numChars;
while ( (numChars = reader.read( buff, 0, buff.length ) ) != -1) {
writer.write( buff, 0, numChars );
}
} catch( IOException ex ) {
throw new IOException("IOException when transferring " + oldLocation.getPath() + " to " + newLocation.getPath());
} finally {
try {
if ( reader != null ){
writer.close();
reader.close();
}
} catch( IOException ex ){
Log.e(getClass().getName(),"Error closing files when transferring " + oldLocation.getPath() + " to " + newLocation.getPath() );
}
}
} else {
throw new IOException("Old location does not exist when transferring " + oldLocation.getPath() + " to " + newLocation.getPath() );
}
}
}
//now copy the below code in activity
StreamingMediaPlayer audioStreamer =
new StreamingMediaPlayer(this,textStreamed,playButton,
streamButton,progressBar);
audioStreamer.startStreaming("your streaming station name",5208, 216);
i think this helps you :)

Starting bbcomm in Java v3 Bloomberg API

When I use the Java Bloomber V3 API it usually works. However, sometimes, especially after a reboot, bbcomm.exe is not running in the background. I can start it manually by running blp.exe, but I wondered if there was a way of doing this via the API?
After talking to the help desk, it turns out that on 64 bit Windows, running under a 64bit JVM bbcomm is not automatically started. This does not happen under 32bit Java - under 32 bit bbcomm automatically runs.
So my solutions are either to wait for the problem to be fixed by Bloomberg (now I understand it) or to check this specific case.
To check the specific case:
if running under a 64 bit windows (System property os.arch)
and if running under a 64bit JVM (System property java.vm.name)
then try and start a session
If this fails, assume bbcomm.exe is not running. Try to run bbcomm.exe using Runtime.exec()
I haven't tested the above yet. It may have exactly the same issues as Bloomberg have with 64bit VMs.
After spending some time with Help Help, it seems that bbcomm gets started either when you use the Excel API or run the API demo. But it does not get started automatically when called from the Java API. Possible ways to start it are:
adding an entry in the registry to automatically start bbcomm on startup: in HKEY_LOCAL_MACHINE\Software\Microsoft\Windows\CurrentVersion\Run add a String value called bbcomm with value C:\blp\API\bbcomm.exe - but that opens a command window which remains visible, so not really an option (and if you close that window it terminates the bbcomm process)
create a batch file START /MIN C:\blp\API\bbcomm.exe and replace the entry in the registry with that (not tested) to call bbcomm silently
manually launch bbcomm from your java code as already suggested. As a reference, I post below the code that I'm using.
private final static Logger logger = LoggerFactory.getLogger(BloombergUtils.class);
private final static String BBCOMM_PROCESS = "bbcomm.exe";
private final static String BBCOMM_FOLDER = "C:/blp/API";
/**
*
* #return true if the bbcomm process is running
*/
public static boolean isBloombergProcessRunning() {
return ShellUtils.isProcessRunning(BBCOMM_PROCESS);
}
/**
* Starts the bbcomm process, which is required to connect to the Bloomberg data feed
* #return true if bbcomm was started successfully, false otherwise
*/
public static boolean startBloombergProcessIfNecessary() {
if (isBloombergProcessRunning()) {
logger.info(BBCOMM_PROCESS + " is started");
return true;
}
Callable<Boolean> startBloombergProcess = getStartingCallable();
return getResultWithTimeout(startBloombergProcess, 1, TimeUnit.SECONDS);
}
private static Callable<Boolean> getStartingCallable() {
return new Callable<Boolean>() {
#Override
public Boolean call() throws Exception {
logger.info("Starting " + BBCOMM_PROCESS + " manually");
ProcessBuilder pb = new ProcessBuilder(BBCOMM_PROCESS);
pb.directory(new File(BBCOMM_FOLDER));
pb.redirectErrorStream(true);
Process p = pb.start();
BufferedReader reader = new BufferedReader(new InputStreamReader(p.getInputStream()));
String line;
while ((line = reader.readLine()) != null) {
if (line.toLowerCase().contains("started")) {
logger.info(BBCOMM_PROCESS + " is started");
return true;
}
}
return false;
}
};
}
private static boolean getResultWithTimeout(Callable<Boolean> startBloombergProcess, int timeout, TimeUnit timeUnit) {
ExecutorService executor = Executors.newSingleThreadExecutor(new ThreadFactory() {
#Override
public Thread newThread(Runnable r) {
Thread t = new Thread(r, "Bloomberg - bbcomm starter thread");
t.setDaemon(true);
return t;
}
});
Future<Boolean> future = executor.submit(startBloombergProcess);
try {
return future.get(timeout, timeUnit);
} catch (InterruptedException ignore) {
Thread.currentThread().interrupt();
return false;
} catch (ExecutionException | TimeoutException e) {
logger.error("Could not start bbcomm", e);
return false;
} finally {
executor.shutdownNow();
try {
if (!executor.awaitTermination(100, TimeUnit.MILLISECONDS)) {
logger.warn("bbcomm starter thread still running");
}
} catch (InterruptedException ex) {
Thread.currentThread().interrupt();
}
}
}
ShellUtils.java
public class ShellUtils {
private final static Logger logger = LoggerFactory.getLogger(ShellUtils.class);
/**
* #return a list of processes currently running
* #throws RuntimeException if the request sent to the OS to get the list of running processes fails
*/
public static List<String> getRunningProcesses() {
List<String> processes = new ArrayList<>();
try {
Process p = Runtime.getRuntime().exec(System.getenv("windir") + "\\system32\\" + "tasklist.exe");
BufferedReader input = new BufferedReader(new InputStreamReader(p.getInputStream()));
String line;
int i = 0;
while ((line = input.readLine()) != null) {
if (!line.isEmpty()) {
String process = line.split(" ")[0];
if (process.contains("exe")) {
processes.add(process);
}
}
}
} catch (IOException e) {
throw new RuntimeException("Could not retrieve the list of running processes from the OS");
}
return processes;
}
/**
*
* #param processName the name of the process, for example "explorer.exe"
* #return true if the process is currently running
* #throws RuntimeException if the request sent to the OS to get the list of running processes fails
*/
public static boolean isProcessRunning(String processName) {
List<String> processes = getRunningProcesses();
return processes.contains(processName);
}
}
In case someone needs help checking/starting bbcomm.exe process from the code hiding console window, this snippet is written in C#; I hope you can easily translate it to Java.
void Main()
{
var processes = Process.GetProcessesByName("bbcomm");
if (processes.Any())
{
Console.WriteLine(processes.First().ProcessName + " already running");
return;
}
var exePath = #"C:\blp\DAPI\bbcomm.exe";
var processStart = new ProcessStartInfo(exePath);
processStart.UseShellExecute = false;
processStart.CreateNoWindow = true;
processStart.RedirectStandardError = true;
processStart.RedirectStandardOutput = true;
processStart.RedirectStandardInput = true;
var process = Process.Start(processStart);
Console.WriteLine(process.ProcessName + " started");
}
bbcomm.exe is automatically started by the V3 API.