You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-06-10 - 18:20

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack7slot7s.osadl.org (updated Wed Jun 10, 2026 12:43:51)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2824210289,9sleep20-21swapper/207:07:022
297029784,9sleep00-21swapper/007:08:540
160629265,23sleep30-21swapper/307:05:183
285928366,13sleep10-21swapper/107:07:291
314799310,5cyclictest0-21swapper/309:32:233
314099280,28cyclictest0-21swapper/110:18:071
314599254,3cyclictest8628-21apt10:35:002
314099240,3cyclictest0-21swapper/110:36:131
314099240,23cyclictest0-21swapper/112:14:211
314099240,20cyclictest0-21swapper/110:58:001
314099230,2cyclictest0-21swapper/109:41:081
3136992322,1cyclictest0-21swapper/011:57:330
3136992322,1cyclictest0-21swapper/011:57:330
314799227,11cyclictest27753-21cut11:05:183
3140992221,1cyclictest0-21swapper/110:12:431
314599210,21cyclictest0-21swapper/211:36:502
314099210,20cyclictest0-21swapper/112:35:271
314099210,19cyclictest0-21swapper/111:14:381
3147992014,4cyclictest0-21swapper/310:45:443
314799200,2cyclictest0-21swapper/310:12:423
314799200,19cyclictest0-21swapper/309:58:433
314099200,20cyclictest0-21swapper/112:24:301
314099200,19cyclictest0-21swapper/111:19:061
313699201,1cyclictest0-21swapper/010:05:140
314799190,1cyclictest0-21swapper/311:36:453
314799190,19cyclictest0-21swapper/312:30:453
314599199,8cyclictest18915-21missed_timers07:40:172
314099193,2cyclictest29153-21diskmemload12:02:361
3140991918,1cyclictest29153-21diskmemload12:16:501
3136991919,0cyclictest0-21swapper/011:41:380
3136991918,1cyclictest0-21swapper/011:14:460
3136991918,1cyclictest0-21swapper/009:05:120
3136991911,5cyclictest0-21swapper/010:24:580
3136991911,5cyclictest0-21swapper/010:24:570
313699190,1cyclictest0-21swapper/010:56:340
313699190,1cyclictest0-21swapper/009:48:290
3147991815,2cyclictest0-21swapper/310:54:083
314799181,2cyclictest0-21swapper/308:40:133
314799180,1cyclictest0-21swapper/311:19:063
314599188,1cyclictest141rcu_preempt10:49:432
314099185,12cyclictest29153-21diskmemload11:32:141
314099180,16cyclictest0-21swapper/111:44:131
314799171,3cyclictest0-21swapper/311:55:253
314799171,3cyclictest0-21swapper/311:55:243
314799170,1cyclictest0-21swapper/311:11:033
314599170,17cyclictest0-21swapper/211:50:562
314099174,10cyclictest25216-21apt-get07:55:121
314099170,17cyclictest29153-21diskmemload09:54:511
314099170,14cyclictest0-21swapper/110:51:411
313699175,8cyclictest31506-21/usr/sbin/munin09:50:100
313699172,8cyclictest12651-21idleruntime-cro07:30:000
313699170,9cyclictest16948-21runrttasks10:40:000
313699170,3cyclictest0-21swapper/009:24:560
314799167,4cyclictest0-21swapper/311:24:593
314799165,10cyclictest0-21swapper/311:44:063
314799162,0cyclictest0-21swapper/310:31:053
3147991615,1cyclictest39-21ksoftirqd/310:09:033
3147991615,1cyclictest0-21swapper/312:06:393
3147991615,1cyclictest0-21swapper/309:50:323
314799161,1cyclictest29153-21diskmemload11:25:303
314799161,15cyclictest0-21swapper/309:46:403
314799160,9cyclictest0-21swapper/307:10:133
314799160,13cyclictest0-21swapper/310:19:043
314599160,16cyclictest0-21swapper/210:37:162
314599160,15cyclictest30245-21ls12:05:162
3140991615,1cyclictest0-21swapper/109:49:031
3140991613,3cyclictest0-21swapper/110:08:351
3140991613,2cyclictest38150irq/131-snd_hda12:26:291
314099160,3cyclictest0-21swapper/110:30:001
314099160,1cyclictest0-21swapper/111:57:481
314099160,1cyclictest0-21swapper/111:57:471
3136991613,3cyclictest13-21ksoftirqd/012:03:490
3147991514,1cyclictest0-21swapper/308:45:013
3147991511,1cyclictest21999-21idleruntime-cro08:55:003
314799150,2cyclictest0-21swapper/311:49:333
314799150,11cyclictest0-21swapper/311:52:443
314599156,6cyclictest29594-21apache209:27:022
314599150,15cyclictest0-21swapper/211:31:462
3140991514,1cyclictest25-21ksoftirqd/109:10:181
3140991514,1cyclictest0-21swapper/109:56:461
314099150,15cyclictest0-21swapper/112:05:451
314099150,15cyclictest0-21swapper/111:54:291
314099150,15cyclictest0-21swapper/111:36:421
314099150,15cyclictest0-21swapper/111:07:521
313699157,7cyclictest22012-21munin-run08:55:000
313699156,9cyclictest13-21ksoftirqd/009:36:420
313699150,1cyclictest0-21swapper/012:20:110
314799140,5cyclictest0-21swapper/312:26:113
314799140,1cyclictest0-21swapper/312:18:163
3145991413,1cyclictest0-21swapper/211:15:552
3145991411,2cyclictest8263-21grep09:55:172
314599140,1cyclictest0-21swapper/211:28:532
314099144,5cyclictest9951-21apt-get08:30:121
3140991413,1cyclictest0-21swapper/108:10:261
313699148,1cyclictest0-21swapper/012:35:130
313699146,7cyclictest5081-21apt-get09:15:010
313699143,6cyclictest18808-21apt-get11:20:000
3136991410,3cyclictest20769-21cut11:20:200
3136991410,2cyclictest7398-21grep07:15:240
3136991410,2cyclictest5369-21sed12:10:000
313699140,7cyclictest1203-21munin-run10:50:000
314799133,8cyclictest24664-21apt-get09:00:123
314799133,6cyclictest0-21swapper/309:26:433
3147991310,2cyclictest3718-21ntpq08:15:213
314799130,1cyclictest0-21swapper/312:20:333
314799130,11cyclictest0-21swapper/310:40:133
314599139,3cyclictest20787-21grep11:40:162
314599139,3cyclictest10593-21tr07:25:132
314599136,7cyclictest1436-21sendmail-mta12:39:212
314599136,3cyclictest225-21systemd-journal07:34:592
314599131,6cyclictest0-21swapper/212:15:012
314599131,3cyclictest0-21swapper/212:23:482
3145991310,2cyclictest5493-21sleep07:15:012
3145991310,2cyclictest31457-21grep12:25:142
3145991310,2cyclictest30851-21wget09:30:132
3145991310,2cyclictest29161-21sleep09:10:012
3145991310,2cyclictest27870-21cat11:05:192
3145991310,2cyclictest26736-21grep10:25:242
3145991310,2cyclictest2477-21grep10:50:172
3145991310,2cyclictest23885-21grep08:55:252
3145991310,2cyclictest23572-21cron10:04:592
3145991310,2cyclictest10628-21ls11:14:592
314599130,1cyclictest0-21swapper/212:17:472
314099130,12cyclictest0-21swapper/111:27:321
314099130,12cyclictest0-21swapper/110:32:451
313699139,3cyclictest25415-21sed10:25:130
313699139,3cyclictest17591-21sh07:40:000
313699133,9cyclictest27750-21grep11:05:190
313699131,6cyclictest7820-21munin-run10:15:010
3136991310,2cyclictest8407-21tr07:20:130
3136991310,2cyclictest15603-21grep08:40:150
313699130,13cyclictest0-21swapper/011:37:070
314799129,2cyclictest8885-21grep07:20:173
314799129,2cyclictest7087-21cron08:25:013
314799129,2cyclictest6456-21sort09:15:163
314799129,2cyclictest6355-21cut12:10:163
314799129,2cyclictest39-21ksoftirqd/311:31:183
314799129,2cyclictest29801-21ssh09:10:123
314799129,2cyclictest26778-21wc07:55:263
314799129,2cyclictest24426-21wc09:00:013
314799129,2cyclictest22216-21cut12:00:163
314799129,2cyclictest19202-21ssh10:20:543
314799129,2cyclictest19202-21ssh10:20:533
314799129,2cyclictest18257-21sh11:00:073
314799129,2cyclictest15599-21ssh09:20:213
314799129,2cyclictest15539-21chrt10:00:013
314799129,2cyclictest15206-21tr12:35:143
314799129,2cyclictest12930-21grep07:30:123
314799124,5cyclictest30261-21apt-get08:05:143
314799120,12cyclictest0-21swapper/310:26:363
314799120,11cyclictest29864-21sshd08:03:343
314799120,11cyclictest0-21swapper/309:41:133
314799120,11cyclictest0-21swapper/307:40:113
314599129,2cyclictest8428-21egrep10:15:112
314599129,2cyclictest7281-21sh09:15:222
314599129,2cyclictest31076-21sed09:10:202
314599129,2cyclictest29520-21cut11:45:172
314599129,2cyclictest28045-21tr08:00:142
314599129,2cyclictest25107-21seq07:55:012
314599129,2cyclictest19502-21bc11:20:102
314599129,2cyclictest17033-21cron08:45:012
314599129,2cyclictest15722-21tail09:40:172
314599128,3cyclictest14455-21awk08:40:002
314599128,1cyclictest10484-21chrt09:36:562
3145991210,2cyclictest32-21ksoftirqd/210:22:392
3145991210,2cyclictest32-21ksoftirqd/210:22:382
3145991210,1cyclictest23989-21grep09:45:172
314099129,2cyclictest5765-21grep08:20:181
314099129,2cyclictest28627-21rm11:45:071
314099129,2cyclictest26209-21cron11:04:591
314099129,2cyclictest22963-21gltestperf08:55:161
314099129,2cyclictest17753-21tr08:45:151
314099129,2cyclictest17593-21apt07:40:001
314099129,2cyclictest15667-21ssh10:00:071
314099128,4cyclictest10137-21taskset07:23:351
314099128,3cyclictest677-21ssh10:49:361
314099127,4cyclictest30637-21expr08:05:141
314099127,2cyclictest7534-21taskset07:15:571
314099124,6cyclictest16065-21ntpq08:40:191
3140991210,1cyclictest25-21ksoftirqd/110:20:221
3140991210,1cyclictest25-21ksoftirqd/110:20:211
313699129,2cyclictest896-21grep10:30:140
313699129,2cyclictest4288-21grep08:15:270
313699129,2cyclictest4112-21tr11:30:150
313699129,2cyclictest31951-21grep08:05:260
313699129,2cyclictest30215-21munin-run09:29:590
313699129,2cyclictest23964-21sed08:55:260
313699129,2cyclictest22698-21dump-pmu-power09:44:590
313699129,2cyclictest19212-21tail11:00:160
313699129,2cyclictest19137-21taskset07:40:190
313699129,2cyclictest11174-21grep08:30:190
313699129,2cyclictest104832sleep009:17:340
313699128,3cyclictest2315-21cron08:15:000
313699128,2cyclictest27319-21sh07:59:590
313699128,2cyclictest25384-21cat10:45:010
313699123,8cyclictest537-21snmpd12:17:560
313699122,2cyclictest1142-21runrttasks10:19:230
313699121,6cyclictest9707-21perf08:30:000
313699120,8cyclictest5616-21taskset11:50:180
314799119,1cyclictest9016-21ssh10:35:093
314799119,1cyclictest8725-21grep08:25:193
314799119,1cyclictest6288-21sh09:35:003
314799119,1cyclictest32661-21wget08:10:133
314799119,1cyclictest28513-21cat09:05:253
314799119,1cyclictest22907-21tr07:50:133
314799119,1cyclictest20513-21tr07:45:163
314799118,2cyclictest15474-21basename07:35:103
314799118,2cyclictest10749-21cut10:55:163
314799118,2cyclictest10520-21apache_processe07:25:113
314599118,2cyclictest760-21head10:10:142
314599118,2cyclictest7198-21sed12:30:142
314599118,2cyclictest31939-21grep08:05:262
314599118,2cyclictest318-21ssh09:50:182
314599118,2cyclictest27978-21ssh12:04:022
314599118,2cyclictest21544-21sed08:50:242
314599118,2cyclictest21160-21ssh10:41:482
314599118,2cyclictest20127-21ssh11:59:272
314599118,2cyclictest20127-21ssh11:59:262
314599118,2cyclictest16236-21grep07:35:162
314599118,2cyclictest13882-21ssh09:20:052
314599118,2cyclictest1145-21cut08:10:182
314599118,2cyclictest10857-21grep10:55:162
314599117,3cyclictest24246-21grep07:50:252
314599110,1cyclictest0-21swapper/210:05:042
314099119,2cyclictest8645-21ssh09:35:301
314099119,2cyclictest23097-21expr07:50:141
314099119,1cyclictest54052sleep107:14:111
314099119,1cyclictest24668-21aten2_r7power_c09:00:121
314099119,1cyclictest12546-21grep08:35:161
314099119,1cyclictest12282-21taskset07:25:401
314099119,1cyclictest10020-21ssh09:17:121
314099118,2cyclictest19437-21ssh11:20:081
314099118,2cyclictest13029-21cut07:30:121
314099117,3cyclictest38150irq/131-snd_hda09:30:181
314099113,7cyclictest23625-21ntp_states09:25:191
3140991110,1cyclictest0-21swapper/112:30:341
3140991110,1cyclictest0-21swapper/110:41:131
314099110,1cyclictest0-21swapper/109:20:231
314099110,10cyclictest27089-21aten2_r7power_e09:05:131
314099110,10cyclictest20617-21expr08:50:151
313699118,2cyclictest5434-21ssh12:10:030
313699118,2cyclictest4759-21ssh10:51:150
313699118,2cyclictest31935-21ssh09:30:190
313699118,2cyclictest3146-21ssh11:29:540
313699118,2cyclictest30307-21ssh11:45:220
313699118,2cyclictest18372-21ssh10:00:540
313699118,2cyclictest14407-21apt12:35:000
313699118,2cyclictest12947-21grep08:35:190
313699118,2cyclictest12263-21ssh09:57:390
313699118,2cyclictest1092-21ssh12:26:050
313699117,3cyclictest6691-21sed08:20:270
313699117,3cyclictest21036-21sed07:45:190
314799109,1cyclictest75472sleep307:16:083
314799109,1cyclictest11311-21tr08:30:213
314799108,1cyclictest13825-21seq08:35:273
314599108,1cyclictest99562sleep207:21:172
314599108,1cyclictest5197-21cat08:20:152
314599108,1cyclictest46732sleep208:19:462
314599108,1cyclictest25262-21grep09:00:152
314599108,1cyclictest20584-21expr07:45:162
314599108,1cyclictest19346-21taskset08:46:352
314599108,1cyclictest18440-21ssh11:00:122
314599103,7cyclictest0-21swapper/208:30:122
314599101,9cyclictest0-21swapper/208:25:122
314599100,7cyclictest0-21swapper/207:15:112
314099109,1cyclictest7965-21expr08:25:141
314099109,1cyclictest21997-21seq07:45:311
314099108,2cyclictest18258-21expr07:40:131
314099108,1cyclictest28255-21cat08:00:161
314099108,1cyclictest2821-21tr08:15:151
313699108,1cyclictest28066-21expr08:00:150
313699108,1cyclictest194592sleep008:48:020
313699107,2cyclictest3970-21grep07:10:150
313699100,9cyclictest26580-21chrt09:02:430
31369997,1cyclictest13819-21grep07:30:170
31369996,2cyclictest23168-21df07:50:150
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional