You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-09-20 - 14:59

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack7slot3s.osadl.org (updated Sun Sep 20, 2026 12:43:40)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
754821740,6sleep01548399cyclictest10:08:020
154839912752,23cyclictest0-21swapper/007:15:300
154839912752,23cyclictest0-21swapper/007:15:300
154839911836,8cyclictest7950irq/56-dwc_otg_10:00:350
486721140,2sleep00-21swapper/010:45:460
198072920,3sleep3381ktimersoftd/307:20:473
101982910,7sleep21548599cyclictest08:25:242
237472880,3sleep00-21swapper/007:30:510
1548599883,31cyclictest7850irq/56-dwc_otg_08:40:402
1548399888,37cyclictest0-21swapper/009:55:370
33152830,4sleep03281-21ssh12:05:540
15484998112,54cyclictest10386-21sh10:10:551
1548499802,63cyclictest26544-21apt-get07:40:371
1548399783,60cyclictest0-21swapper/007:10:270
1548499773,66cyclictest0-21swapper/107:10:271
1548499761,42cyclictest5381-21ssh10:05:391
15484997613,44cyclictest20024-21unixbench_singl11:45:551
1548499752,64cyclictest0-21swapper/110:45:051
1548499734,55cyclictest1244-21runrttasks10:00:351
1548499724,58cyclictest11373-21sh09:30:541
1548499721,6cyclictest18476-21ssh10:21:231
1548499721,49cyclictest31640-21apt-get11:20:361
1548499721,49cyclictest27125-21if_eth011:55:441
1548499721,43cyclictest17131-21ssh11:42:511
1548699716,8cyclictest7750irq/56-dwc_otg12:05:263
1548499712,55cyclictest0-21swapper/111:25:361
15484997117,40cyclictest11031-21tr11:35:381
15483997125,21cyclictest0-21swapper/010:40:360
1548499702,52cyclictest7850irq/56-dwc_otg_09:00:241
1548499702,52cyclictest7850irq/56-dwc_otg_09:00:241
1548499702,50cyclictest7950irq/56-dwc_otg_12:30:531
1548499701,55cyclictest28560-21df_inode07:45:401
1548499701,51cyclictest524-21ssh12:03:051
1548499701,51cyclictest15531-21ssh09:36:171
1548499701,47cyclictest32126-21ssh09:57:571
1548499693,48cyclictest27718-21ssh11:15:341
1548499691,58cyclictest20188-21snmpd09:22:131
1548499691,58cyclictest11735-21ssh10:55:231
1548499691,57cyclictest19810-21ssh11:05:261
1548499691,54cyclictest7760-21ssh10:50:171
1548499691,50cyclictest1463-21ssh09:19:271
1548499691,47cyclictest7750irq/56-dwc_otg10:55:461
1548499691,47cyclictest7750irq/56-dwc_otg10:55:461
1548499691,46cyclictest17870-21/usr/sbin/munin07:15:441
1548499691,46cyclictest17870-21/usr/sbin/munin07:15:441
1548399691,36cyclictest0-21swapper/009:05:380
15486996810,13cyclictest0-21swapper/312:37:373
1548499684,42cyclictest7750irq/56-dwc_otg12:14:131
1548499682,55cyclictest24990-21ssh09:49:331
1548499671,46cyclictest15753-21ssh10:18:461
1548499671,42cyclictest0-21swapper/112:39:271
15483996724,21cyclictest32477-21cat10:40:250
1548399671,12cyclictest0-21swapper/009:35:370
1548699665,6cyclictest7950irq/56-dwc_otg_12:15:543
1548699665,4cyclictest7950irq/56-dwc_otg_08:55:393
1548699665,4cyclictest7950irq/56-dwc_otg_08:55:393
1548699662,6cyclictest0-21swapper/309:57:433
1548499664,42cyclictest0-21swapper/111:10:461
15484996618,44cyclictest20505-21sed08:50:531
1548499661,48cyclictest22506-21runrttasks10:26:381
1548499661,48cyclictest22506-21runrttasks10:26:381
15483996625,8cyclictest218392sleep012:30:370
1548399662,39cyclictest30538-21latency_hist09:15:250
15483996616,28cyclictest6634-21latency12:10:480
1548699651,20cyclictest14509-21latency_hist11:40:253
1548499653,47cyclictest7850irq/56-dwc_otg_11:30:391
1548499652,52cyclictest0-21swapper/112:29:491
1548499652,47cyclictest25196-21diskstats09:05:411
1548499651,48cyclictest13312-21sh12:19:541
1548499651,44cyclictest22660-21ssh11:50:361
1548499651,42cyclictest25868-21ssh09:50:361
15483996521,31cyclictest7850irq/56-dwc_otg_12:25:250
1548699642,5cyclictest7850irq/56-dwc_otg_11:50:443
1548699641,5cyclictest7750irq/56-dwc_otg11:25:433
1548699641,13cyclictest7950irq/56-dwc_otg_07:45:503
1548499645,40cyclictest7950irq/56-dwc_otg_10:34:591
1548499642,46cyclictest0-21swapper/108:16:331
1548499641,43cyclictest31944-21sh10:39:501
1548699633,5cyclictest0-21swapper/310:55:483
1548699633,5cyclictest0-21swapper/310:55:483
1548699632,18cyclictest16981-21switchtime11:00:523
1548699632,12cyclictest7950irq/56-dwc_otg_09:00:473
1548499632,49cyclictest18811-21ssh11:03:541
1548499631,50cyclictest20188-21snmpd08:50:001
1548499631,50cyclictest0-21swapper/109:44:051
1548499631,45cyclictest24744-21munin-run09:05:241
15483996347,9cyclictest0-21swapper/011:55:310
1548399631,24cyclictest0-21swapper/010:34:090
234992620,2sleep00-21swapper/009:46:340
1548699624,6cyclictest7750irq/56-dwc_otg11:45:473
1548699624,5cyclictest20081-21missed_timers08:50:473
1548499623,48cyclictest28287-21ssh09:11:041
1548499621,49cyclictest24160-21taskset07:31:201
1548499621,45cyclictest15939-21chrt12:22:191
1548499621,41cyclictest0-21swapper/108:37:141
1548499621,40cyclictest1723-21apt-get12:05:271
1548399625,28cyclictest0-21swapper/011:10:380
1548399622,48cyclictest0-21swapper/009:30:430
222112610,1sleep00-21swapper/007:25:550
1548699619,10cyclictest0-21swapper/311:09:013
1548499612,40cyclictest7750irq/56-dwc_otg08:05:231
1548499611,37cyclictest17263-21dhclient-script08:44:521
15483996118,5cyclictest0-21swapper/010:55:380
15483996118,5cyclictest0-21swapper/010:55:380
1548399611,55cyclictest21744-21ssh10:25:510
1548399611,55cyclictest21744-21ssh10:25:510
1548699608,45cyclictest0-21swapper/311:10:533
1548699601,17cyclictest8801-21missed_timers10:50:473
1548499601,47cyclictest1016-21runrttasks08:00:011
1548499601,46cyclictest0-21swapper/107:29:031
1548499601,45cyclictest25918-21users07:35:561
1548499601,45cyclictest25918-21users07:35:561
1548499601,42cyclictest0-21swapper/108:11:481
15483996033,8cyclictest29489-21latency_hist12:40:250
15483996025,23cyclictest9-21ksoftirqd/011:25:360
1548399601,27cyclictest8709-21ls08:20:390
1548699594,4cyclictest7850irq/56-dwc_otg_11:15:593
1548699592,21cyclictest0-21swapper/312:27:323
1548699591,9cyclictest0-21swapper/308:47:343
15486995918,6cyclictest26985-21ssh09:50:533
1548699591,18cyclictest0-21swapper/310:00:453
1548499591,47cyclictest9982-21taskset08:24:071
1548499591,39cyclictest12206-21aten_rbpower_en08:30:411
15483995928,7cyclictest27120-21grep09:50:540
1548399592,16cyclictest0-21swapper/009:40:510
15483995919,33cyclictest2812-21kernelversion09:20:440
1548699587,4cyclictest7950irq/56-dwc_otg_12:05:423
1548699582,15cyclictest0-21swapper/310:16:383
15486995818,8cyclictest2557-21fschecks_count09:20:413
1548599586,9cyclictest7750irq/56-dwc_otg09:55:372
1548499584,40cyclictest7950irq/56-dwc_otg_09:25:501
1548499582,45cyclictest7950irq/56-dwc_otg_08:27:001
1548499581,42cyclictest0-21swapper/107:22:061
1548399587,33cyclictest0-21swapper/010:15:380
15483995824,14cyclictest7750irq/56-dwc_otg08:30:240
1548399581,38cyclictest30961-21idleruntime12:00:430
1548699577,7cyclictest0-21swapper/309:05:353
1548699577,4cyclictest7750irq/56-dwc_otg09:14:193
1548699576,11cyclictest7850irq/56-dwc_otg_10:08:443
1548699574,13cyclictest7950irq/56-dwc_otg_10:45:303
1548699572,6cyclictest24980-21df_inode10:30:413
1548699571,6cyclictest0-21swapper/310:20:423
15486995711,4cyclictest7850irq/56-dwc_otg_10:27:573
15486995711,4cyclictest7850irq/56-dwc_otg_10:27:573
1548499571,34cyclictest31469-21taskset07:52:361
15483995716,37cyclictest101ktimersoftd/011:45:370
1548699561,14cyclictest0-21swapper/311:34:533
15485995620,11cyclictest12087-21apt-get10:55:382
15485995620,11cyclictest12087-21apt-get10:55:382
1548499561,4cyclictest44222sleep108:07:261
1548399564,21cyclictest0-21swapper/012:25:360
1548399563,8cyclictest0-21swapper/007:40:340
1548399562,31cyclictest0-21swapper/011:45:000
146552563,7sleep00-21swapper/007:05:560
1548699555,6cyclictest21850-21ssh09:45:363
15486995510,4cyclictest0-21swapper/312:33:533
15485995517,23cyclictest7750irq/56-dwc_otg12:34:422
15483995528,12cyclictest7850irq/56-dwc_otg_11:00:390
15483995522,16cyclictest0-21swapper/008:10:480
15483995521,19cyclictest6597-21fschecks_time09:25:450
15483995520,21cyclictest0-21swapper/010:50:290
15483995519,17cyclictest10456-21if_wlan012:15:440
15483995513,17cyclictest9474-21sh11:33:080
1548699545,6cyclictest39-21ksoftirqd/309:30:483
15486995419,5cyclictest11109-21processes08:25:513
1548699541,4cyclictest7750irq/56-dwc_otg11:25:243
1548599548,29cyclictest0-21swapper/208:55:392
1548599548,29cyclictest0-21swapper/208:55:392
15483995433,10cyclictest9-21ksoftirqd/009:00:240
15483995433,10cyclictest9-21ksoftirqd/009:00:240
15483995428,15cyclictest7850irq/56-dwc_otg_10:20:380
15483995427,10cyclictest7750irq/56-dwc_otg11:05:430
15483995421,16cyclictest19561-21wget08:50:390
1548399541,3cyclictest0-21swapper/011:50:410
1548699538,28cyclictest0-21swapper/307:15:303
1548699538,28cyclictest0-21swapper/307:15:303
1548699534,4cyclictest0-21swapper/310:11:443
1548699532,16cyclictest0-21swapper/307:36:203
1548699532,16cyclictest0-21swapper/307:36:203
1548599538,29cyclictest0-21swapper/212:25:312
1548599532,13cyclictest6077-21ssh12:10:382
15485995316,13cyclictest26242-21apt-get11:55:292
15485995313,12cyclictest0-21swapper/211:20:342
15485995311,19cyclictest23798-21timerwakeupswit11:50:552
1548399535,25cyclictest0-21swapper/007:50:530
1548399532,7cyclictest16289-21grep08:40:470
15483995324,15cyclictest11806-21taskset11:35:500
15483995319,22cyclictest0-21swapper/007:40:200
15483995319,22cyclictest0-21swapper/007:40:200
1548399531,28cyclictest0-21swapper/009:00:390
1548399531,24cyclictest3130750ssh09:15:450
1548699528,28cyclictest0-21swapper/307:30:303
1548699522,9cyclictest0-21swapper/308:00:363
1548599528,29cyclictest0-21swapper/209:10:372
1548599528,28cyclictest0-21swapper/208:45:302
1548599526,26cyclictest4388-21/usr/sbin/munin11:25:552
1548399525,37cyclictest11-21rcu_preempt11:20:330
15483995225,11cyclictest7950irq/56-dwc_otg_07:50:240
15483995221,17cyclictest7950irq/56-dwc_otg_08:00:240
1548599519,30cyclictest0-21swapper/208:35:382
1548599518,29cyclictest0-21swapper/210:05:372
1548599517,31cyclictest26763-21ssh10:32:182
1548599517,25cyclictest20699-21sh11:05:502
15485995124,22cyclictest15487-21runrttasks07:10:282
15483995130,13cyclictest9-21ksoftirqd/010:10:370
15483995127,13cyclictest0-21swapper/008:45:360
1548399512,25cyclictest28158-21ssh11:15:420
1548399511,29cyclictest16203-21systemd-journal08:05:420
1548699501,13cyclictest7950irq/56-dwc_otg_10:45:103
1548599509,27cyclictest0-21swapper/210:40:302
1548599508,28cyclictest0-21swapper/209:30:382
1548599508,27cyclictest0-21swapper/210:45:382
15485995013,15cyclictest7950irq/56-dwc_otg_11:15:532
1548399501,5cyclictest6785-21grep08:15:390
1548599499,23cyclictest573-21ssh12:03:112
1548599497,23cyclictest25536-21runrttasks09:05:452
1548599495,20cyclictest7750irq/56-dwc_otg09:50:302
1548399492,38cyclictest0-21swapper/008:37:310
15483994916,24cyclictest0-21swapper/007:23:120
15483994914,24cyclictest0-21swapper/008:30:420
1548699484,16cyclictest14841-21ssh11:40:363
1548599488,26cyclictest0-21swapper/212:05:312
1548599484,27cyclictest0-21swapper/209:00:312
15485994830,9cyclictest7950irq/56-dwc_otg_07:15:412
15485994830,9cyclictest7950irq/56-dwc_otg_07:15:412
15485994823,7cyclictest1180-21runrttasks12:20:492
15485994817,10cyclictest1237-21apt-get10:00:382
1548699476,9cyclictest0-21swapper/312:13:493
1548599478,20cyclictest24865-21timerandwakeup11:10:532
1548599475,22cyclictest8348-21uptime11:30:552
15485994710,27cyclictest10260-21/usr/sbin/munin10:10:542
1548399472,26cyclictest0-21swapper/008:00:390
1548699468,24cyclictest0-21swapper/309:15:383
1548699461,6cyclictest17949-21ssh09:40:373
1548599469,14cyclictest28247-21latency_hist07:45:242
1548599468,25cyclictest0-21swapper/207:30:382
1548599466,30cyclictest7955-21chrt08:16:262
1548599465,19cyclictest23993-21ssh10:29:362
1548599465,19cyclictest23993-21ssh10:29:362
15485994629,5cyclictest7950irq/56-dwc_otg_10:16:302
15485994618,12cyclictest7850irq/56-dwc_otg_09:44:332
15485994611,22cyclictest28565-21/usr/sbin/munin07:45:412
15485994610,13cyclictest16528-21/usr/sbin/munin11:00:452
1548699455,14cyclictest7850irq/56-dwc_otg_07:10:283
15486994512,13cyclictest270652latency07:40:443
1548599459,27cyclictest25724-21timerwakeupswit07:35:542
1548599459,27cyclictest25724-21timerwakeupswit07:35:542
1548599457,26cyclictest0-21swapper/209:20:372
1548599455,24cyclictest16950-21ssh09:39:062
15485994527,10cyclictest7850irq/56-dwc_otg_10:50:362
15485994511,20cyclictest12081-21ssh12:17:282
1548599445,24cyclictest7950irq/56-dwc_otg_11:41:012
1548599443,27cyclictest19992-21rm10:24:252
1548699438,27cyclictest0-21swapper/307:55:393
15485994323,11cyclictest7950irq/56-dwc_otg_11:38:062
15485994319,5cyclictest7750irq/56-dwc_otg10:37:122
1548599431,26cyclictest7950irq/56-dwc_otg_09:49:592
15485994310,22cyclictest0-21swapper/212:39:172
1548699421,28cyclictest27068-21ls11:55:433
1548599427,21cyclictest18739-21ssh11:45:352
15485994215,8cyclictest7850irq/56-dwc_otg_07:53:512
15486994131,6cyclictest381ktimersoftd/307:25:373
15485994122,11cyclictest32454-21taskset09:16:352
308122400,10sleep30-21swapper/310:37:353
15486994018,20cyclictest1411050wget09:35:403
1548599408,17cyclictest0-21swapper/208:50:382
1548599408,17cyclictest0-21swapper/208:10:392
15485994024,9cyclictest7950irq/56-dwc_otg_09:27:432
1548599391,25cyclictest10516-21ls08:25:402
1548699382,4cyclictest0-21swapper/312:20:363
1548599385,5cyclictest7950irq/56-dwc_otg_07:30:252
15485993813,11cyclictest7950irq/56-dwc_otg_08:02:242
15485993810,15cyclictest7750irq/56-dwc_otg08:05:522
63552370,2sleep30-21swapper/309:25:413
1548699372,18cyclictest0-21swapper/308:30:363
1548599372,22cyclictest0-21swapper/207:24:442
1548599371,32cyclictest13892-21ls08:35:252
136672365,16sleep30-21swapper/307:05:393
1548599342,14cyclictest799-21chrt07:57:202
1548699338,13cyclictest0-21swapper/308:14:453
1548699328,5cyclictest7950irq/56-dwc_otg_08:40:483
1548699326,11cyclictest7750irq/56-dwc_otg08:05:503
1548699324,7cyclictest7950irq/56-dwc_otg_08:15:543
1548699322,19cyclictest0-21swapper/308:24:123
15486993123,5cyclictest30208-21cut07:50:383
1548699312,18cyclictest0-21swapper/308:35:503
142032295,13sleep114236-21/usr/sbin/munin07:05:491
142372277,11sleep213655-21/usr/sbin/munin07:05:502
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional