I am responsible for iOS automation at my company and I am working with Swift in XCTest. My company is in the midst of converting to Swift 3 and in the interim I've converted my tests to Swift 2.3 until we've completed the conversion. The automation has slowed to an unacceptable speed. So bad that if we were at the beginning again and choosing frameworks XCTest would have been disqualified. My test times have gone up by almost a third in this transitional swift conversion. Trying to uncover what the issue was this link was helpful:
Swift 3 Compile Time Incredibly Slow
We applied final to all class models to deal with lazy vars which gained us 6 seconds per test! Unfortunately that doesn't bring it back to its original speed :(
As those of us in the mobile automation world know using NSPredicate as recommended by #Joe Masilotti is the best way to wait for elements
https://stackoverflow.com/users/384110/joe-masilotti
I commented out a test that used a custom func waitForElement () that included let existsPredicate = NSPredicate(format: "exists == true"). The same test now gained an additional 3 seconds! So a total of 9 seconds is substantial, but unfortunately this is still not back to the original speed. I hope that expresses my words on how slow swift 2.3 has become.
I am going to trying to convert only the test targets to swift 3 and see if that helps. My theory on that is Xcode 8 is optimized for swift 3 not swift 2.3
My question is, is there another way to write NSpredicate / waitForElement or hittable method in an extension that would not slow down my tests?
The iOS 10 simulator is significantly more resource-intensive than the iOS 9 simulator was.
If you want to run your tests faster, switch to the iOS 9 simulator, however you will need to put up with the slower tests in order to test on the current version of iOS.
Make sure you have a high-spec Mac as your CI server - people often make the mistake of using a base-spec Mac Mini, which is terribly slow at running tests. More power can help a lot.
Related
Is Rebol 3 really different from Rebol 2 and Red-Lang. Is it finished ?
I was in the same boat as you before, hopefully, things are much clearer now. (Can't add to that one as it is closed)
As for finished (usable in production), only Rebol 2 is stable and mature (I myself use it, having only started a few months ago)
In order of easiest to hardest to get started:
Rebol 2:
Pros:
easy to get started (single binary)
stable, mature, full featured
has view (GUI)
lots of documentation
examples at rebol.net
lots of compatible libraries at rebol.org
has a large user base (still!)
Cons:
no active development (version I use is from 2011)
deployment is harder (need commercial SDK for native binaries, but can work around)
no native gui (might not be a problem)
Red:
(based on Rebol 2)
(community on gitter.im)
Pros:
easy to get started (single binary)
dead simple deployment (native binaries)
has native GUI (view and draw, still in development)
active development
Red/System (low level actual alternative to C, it is written in itself/self-hosted)
Cons:
documentation work in progress
not everything is working
small chance of breaking (due to being in alpha)
Ren-C:
(based on Rebol 3)
(community here on stackoverflow chat)
There are many branches of rebol 3:
This question gives a better overview. I chose "Ren-C" because it seems the most actively developed
Note: I haven't actually used "Ren-C", but only other rebol 3 binaries, so refer to the other questions and take this with a grain of salt, but it should be pretty similar to Red in terms of development and community
Pros:
more experimental than red?
active development
written in c/c++
other Rebol 3 (GUI) branches use it as upstream
these GUI versions are used commercially and in production
Cons:
more experimental than red?
harder to get started (compile from source)
written in c/c++
documentation?
based on rebol 3 so less compatible with rebol 2(?)
(actually, there seems to be a porting guide)
would probably be eventually merged into red(?)
As far I know, R3 isn't finished & has bugs. I don't think anyone works on Rebol 3 using that name.
HostileFork & other people are working on C implementations, named Ren/C as far I remember.
Ren/C & Red is work in progress - anything can change.
All 4 languages are very similar but you will find some differences from time to time.
For example:
in Rebol 3 request-file returns file not block of files as in Rebol 2
you can make "a function, making all words found in body local" (I think
Rebol 3 and Ren/c has something like this too)
they are working on parse, so you can expect something "better"
I have a lot of problems with xcode 6 at the moment. For me it is nearly unusable.
- performance during building the code and also errorchecking, indexing ...
- autocomplition doesn't work sometimes.
- redirecting to functions works 20% of time (clicking on function or variable to redirect to code). Sometimes it works sometimes not.
I guess the problems went with growing of my apps code.
So I ask me if I should go for XCode 7, which promises a lot for solving my issues. Then I heard about migration problems too.
Are there any experienced developers which went from xcode6 to sxcode7 and do they suggest to do this ?
Close Xcode. Rename the app "Xcode6". Install Xcode 7. Try it and see if it fixes your problems.
I've just installed ios7 on my iPhone, however I have not yet downloaded xCode5, so I've been producing my archives using iOS6 SDK and using TestFlight to test them on my phone. When running my app on iOS7, I'm noticing a lot of bugs that I didn't see in iOS6 (some that are unpredictable and very difficult to fix). I've been avoiding downloading xCode7 because I'm not yet ready to make the commitment to the new UI elements, but I'm wondering if I did produce my app using the iOS7 SDK, is it possible some of these bugs could be eliminated?
This is a very broad question. iOS 7, despite its advanced beta count, is still very much a work in progress. There are bugs that should be reported to Apple that should not happen. But beyond that, there are changes in the internal API that influence how the app behaves. Apple has done its best to try and preserve SDK 6 apps as much as possible, but there is breaking API which can cause crashes. One example off the top of my head is the class cluster they are now using with ABPersonViewController. Subclassing that in iOS 6 works fine, but in iOS 7, even when compiled with SDK 6, causes a crash in most cases. These issues can be resolved even with Xcode 4.6 and SDK 6.
Compiling with SDK 7 may help you fix some issues, but it will come with a plethora of issues of its own. Depending on how complex your view hierarchy is, you may have to invest a considerable amount of effort to support the new API and functionality. You don't really have a choice, as this is the future, but you should be prepared for this, and arrange your schedule accordingly.
After reading:
iphone: submit app with iOS 5 and XCode 4.2?
I realize that using ARC and expecting to release an app to the app store with this technology might have been a bit short-sighted of me. What is the best course of action for me?
Use Xcode 4 and redo the memory management
Wait for Xcode 4.2 to be released
Some other magical way that will solve all my issues and make my dreams come true
Thanks for the advice.
Update: Just in case anyone wandered on to this, it was while Xcode 4.2 was still in beta. End result: don't use beta features unless you're ok with waiting for the final release.
So, first, there is no Santa Claus. That leaves us with:
How urgently do you want to ship?
Xcode 4.2 is going to ship with iOS 5. iOS 5 is likely to ship around September. It could be later. So can you sit on your app for that long? If so, eh, keep polishing, maybe work on some additional features, do an exhaustive run of QA, then when the GM is ready you'll be all set.
If you want to be the master of your own destiny and ship on your terms, install Xcode 4.1 and run the static analyzer to get the full measure of your missing memory management, spend a few hours fixing things up, then profile and test the hell out of it just to be sure. Read up on the Leaks instrument, along with NSZombieEnabled, should you run into unexpected memory issues (dunno how big your project is, but I'm sure they can build up with the luxury of ARC snatched away).
I'm currently using OCUnit that ships with Xcode 3.2.4 for doing unit testing of my application. My workflow is often to set some break points in a failing unittestin order to quickly inspect the state. I'm using Apple's OCUnit setup:
http://developer.apple.com/library/ios/#documentation/Xcode/Conceptual/iphone_development/135-Unit_Testing_Applications/unit_testing_applications.html
but the setup from above gives me some headaches. Apple distinguish between Application tests and Logic tests. As I see it:
You cannot debug logic tests. It's as if they're invisibly run when you build your project.
You can debug application tests, but you have to run these on the device and not the simulator (what is the reason for this?)
This means that everything moves kind of slowly with my current workflow. Any hints on getting app tests to run on the simulator? Or any pin pointers to another test-framework?
Would eg. google-toolbox-for-mac work better in general or for my specific needs?
Also, general comments on using breakpoints in the unit tests are welcome! :-)
I have used the Google Toolbox testing rig in the past and it worked fine, it ran both on the Simulator and the device and I could debug my tests if I wanted to. Recently I got fed up with bundling so much code with each of my projects and tried the Apple way.
I also find the logic/app tests split weird, especially as I can’t find any way to debug the logic tests now. (And if you’re using some parts of AVFoundation that won’t build for Simulator, you are apparently out of luck with logic tests completely, which seems strange.) One of the pros is that I can run the logic tests quickly during build.
I guess this does not help you that much – the point is that you can debug the tests under GTM. And you might also want to check out this related question.
I know this isn't really a good answer, nor is it completely helpful to your cause. But I've been using NSLog to run my unit tests (if nothing outputs to the console, then success). When I comment out my tests method then the tests wouldn't run. I found this much more predictable and reliable than OCUnit. I'd much rather use a real true unit tester, but it was too frustrating to deal with the often strange errors that could occur from OCUnit and also the other shortfalls/lack of features you describe above.