You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-03-27 - 03:58

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #5

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackeslot5s.osadl.org (updated Fri Mar 27, 2026 00:44:23)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
10256992520,4cyclictest0-21swapper/222:24:502
10256992517,0cyclictest0-21swapper/222:35:042
1025699251,1cyclictest271rcuc/200:25:182
1025699250,1cyclictest21893-21sendmail-msp20:40:012
10251992523,1cyclictest0-21swapper/100:20:231
1025199250,24cyclictest0-21swapper/123:15:041
1025199250,24cyclictest0-21swapper/100:05:581
1025199250,1cyclictest4302-21diskmemload21:19:131
10247992524,1cyclictest4302-21diskmemload22:37:390
1024799250,24cyclictest30055-21ssh00:29:130
1024799250,24cyclictest0-21swapper/022:54:140
1026099240,1cyclictest10363-21ssh21:16:383
1026099240,0cyclictest0-21swapper/300:04:283
10256992423,1cyclictest28-21ksoftirqd/223:45:012
10256992422,1cyclictest8350-21cron22:30:012
1025699240,24cyclictest0-21swapper/221:58:152
1025699240,23cyclictest6008-21idleruntime-cro21:50:002
1025699240,23cyclictest0-21swapper/223:08:442
1025699240,23cyclictest0-21swapper/200:13:032
1025699240,1cyclictest0-21swapper/221:24:512
10251992420,0cyclictest151rcu_preempt00:37:371
1025199240,23cyclictest0-21swapper/123:41:501
1025199240,23cyclictest0-21swapper/123:20:581
1025199240,23cyclictest0-21swapper/122:17:421
1025199240,1cyclictest0-21swapper/121:21:541
1025199240,0cyclictest0-21swapper/100:27:261
10247992424,0cyclictest62550irq/127-eno121:56:520
10247992421,2cyclictest0-21swapper/023:43:030
1024799240,2cyclictest20583-21rm23:15:080
1024799240,24cyclictest4302-21diskmemload23:02:200
1024799240,23cyclictest9862-21dump-pmu-power00:10:000
1024799240,23cyclictest4302-21diskmemload00:35:060
1024799240,23cyclictest28456-21ssh21:34:230
1024799240,23cyclictest19639-21ssh21:25:330
1024799240,23cyclictest0-21swapper/022:00:290
1024799240,23cyclictest0-21swapper/021:10:170
10260992323,0cyclictest7707-21ssh21:52:273
10260992323,0cyclictest0-21swapper/323:40:403
10260992323,0cyclictest0-21swapper/323:14:423
10260992323,0cyclictest0-21swapper/322:40:513
10260992323,0cyclictest0-21swapper/322:06:013
10260992323,0cyclictest0-21swapper/321:22:023
10260992323,0cyclictest0-21swapper/300:13:493
1026099230,23cyclictest0-21swapper/323:31:153
1026099230,23cyclictest0-21swapper/322:58:543
1026099230,23cyclictest0-21swapper/322:53:163
1026099230,23cyclictest0-21swapper/322:45:513
1026099230,23cyclictest0-21swapper/321:10:553
1026099230,23cyclictest0-21swapper/300:25:053
1026099230,23cyclictest0-21swapper/300:17:173
1026099230,22cyclictest31359-21ssh23:26:353
1026099230,22cyclictest30715-21sh21:37:463
1026099230,22cyclictest27089-21ssh23:22:413
1026099230,22cyclictest21893-21sendmail-msp20:40:013
1026099230,0cyclictest0-21swapper/322:36:503
1026099230,0cyclictest0-21swapper/322:27:243
1026099230,0cyclictest0-21swapper/322:10:133
1026099230,0cyclictest0-21swapper/322:10:133
10256992323,0cyclictest0-21swapper/222:55:162
10256992323,0cyclictest0-21swapper/221:43:132
10256992323,0cyclictest0-21swapper/200:35:352
10256992323,0cyclictest0-21swapper/200:21:032
10256992322,1cyclictest28-21ksoftirqd/223:01:472
10256992322,1cyclictest28-21ksoftirqd/222:15:042
1025699232,1cyclictest28-21ksoftirqd/223:45:412
1025699232,1cyclictest28-21ksoftirqd/223:18:142
10256992321,2cyclictest4302-21diskmemload21:51:112
10256992321,2cyclictest27881-21ssh21:33:182
10256992321,1cyclictest30740-21ssh23:25:362
10256992321,1cyclictest271rcuc/221:15:002
10256992321,1cyclictest0-21swapper/222:13:152
10256992321,1cyclictest0-21swapper/222:13:152
1025699230,23cyclictest0-21swapper/223:53:312
1025699230,23cyclictest0-21swapper/222:09:062
1025699230,23cyclictest0-21swapper/221:35:202
1025699230,22cyclictest5640-21sh00:05:522
1025699230,22cyclictest4302-21diskmemload23:35:052
1025699230,22cyclictest313-21systemd-journal21:30:012
1025699230,22cyclictest271rcuc/222:30:322
1025699230,22cyclictest19955-21ssh00:19:372
1025699230,0cyclictest0-21swapper/223:23:152
10251992323,0cyclictest0-21swapper/123:11:271
10251992323,0cyclictest0-21swapper/122:35:381
10251992321,1cyclictest0-21swapper/122:25:021
10251992321,1cyclictest0-21swapper/121:53:471
10251992321,1cyclictest0-21swapper/121:37:511
1025199230,2cyclictest23151-21sh22:44:571
1025199230,23cyclictest0-21swapper/123:57:561
1025199230,23cyclictest0-21swapper/122:12:481
1025199230,23cyclictest0-21swapper/122:12:481
1025199230,22cyclictest5095-21sh21:48:261
1025199230,22cyclictest4302-21diskmemload23:27:071
1025199230,22cyclictest0-21swapper/123:37:031
1025199230,22cyclictest0-21swapper/123:01:581
1025199230,22cyclictest0-21swapper/122:50:561
1025199230,1cyclictest24081-21sh23:52:291
1025199230,0cyclictest0-21swapper/123:46:391
1025199230,0cyclictest0-21swapper/122:48:031
1025199230,0cyclictest0-21swapper/122:32:351
1025199230,0cyclictest0-21swapper/122:20:191
1025199230,0cyclictest0-21swapper/100:15:051
10247992323,0cyclictest0-21swapper/023:50:090
10247992323,0cyclictest0-21swapper/023:29:050
10247992323,0cyclictest0-21swapper/000:12:060
10247992321,1cyclictest0-21swapper/023:56:540
10247992320,2cyclictest23669-21ssh22:45:310
1024799231,0cyclictest0-21swapper/023:32:010
1024799230,23cyclictest4302-21diskmemload22:25:590
1024799230,23cyclictest0-21swapper/023:20:170
1024799230,23cyclictest0-21swapper/021:44:160
1024799230,22cyclictest6342-21rm21:50:100
1024799230,22cyclictest31197-21ssh21:38:400
1024799230,22cyclictest21898-21perf20:40:010
1024799230,22cyclictest2168-21ssh22:57:270
1024799230,22cyclictest11898-21ssh23:07:090
1024799230,0cyclictest0-21swapper/023:46:150
1024799230,0cyclictest0-21swapper/022:30:420
1024799230,0cyclictest0-21swapper/000:20:260
1024799230,0cyclictest0-21swapper/000:17:370
10260992222,0cyclictest0-21swapper/323:35:263
10260992222,0cyclictest0-21swapper/322:24:373
10260992222,0cyclictest0-21swapper/322:15:153
10260992222,0cyclictest0-21swapper/321:30:013
10260992220,1cyclictest3273-21diskmemload21:45:213
10260992220,1cyclictest19078-21apt-get20:35:013
10260992219,2cyclictest24263-21cron20:45:013
1026099220,2cyclictest24277-21ssh23:18:273
1026099220,22cyclictest22227-21sh23:49:393
1026099220,22cyclictest15224-21ssh22:01:183
1026099220,22cyclictest0-21swapper/300:20:213
1026099220,21cyclictest25532-21apt-get19:43:023
1026099220,1cyclictest2285-21rm21:44:123
1026099220,1cyclictest2278-21ssh00:33:333
1026099220,0cyclictest4302-21diskmemload00:39:413
10256992222,0cyclictest0-21swapper/200:31:492
1025699220,21cyclictest4692-21munin-run20:05:012
1025699220,21cyclictest4302-21diskmemload22:50:482
1025699220,21cyclictest361-21sh00:00:242
1025699220,21cyclictest24157-21ssh22:46:182
1025699220,21cyclictest0-21swapper/223:55:392
1025699220,21cyclictest0-21swapper/223:10:502
1025699220,21cyclictest0-21swapper/222:44:552
1025699220,21cyclictest0-21swapper/222:00:322
1025699220,21cyclictest0-21swapper/219:23:012
1025699220,1cyclictest4302-21diskmemload21:17:212
10251992222,0cyclictest4302-21diskmemload21:29:341
10251992221,1cyclictest4302-21diskmemload21:57:111
10251992221,1cyclictest11554-21ssh23:06:361
10251992220,1cyclictest4663-21sh20:05:011
10251992220,1cyclictest24301-21apt-get20:45:021
10251992220,1cyclictest0-21swapper/100:30:591
10251992220,1cyclictest0-21swapper/100:10:051
1025199220,22cyclictest0-21swapper/123:31:591
1025199220,22cyclictest0-21swapper/121:42:131
1025199220,21cyclictest7054-21sh20:10:001
1025199220,21cyclictest4302-21diskmemload21:13:291
1025199220,21cyclictest2202-21sh20:00:011
1025199220,21cyclictest0-21swapper/122:05:051
1025199220,21cyclictest0-21swapper/121:30:541
1025199220,21cyclictest0-21swapper/100:00:561
1025199220,1cyclictest15985-21sh22:02:381
1025199220,1cyclictest0-21swapper/122:55:401
10247992222,0cyclictest4302-21diskmemload23:10:590
10247992222,0cyclictest0-21swapper/022:12:150
10247992222,0cyclictest0-21swapper/022:12:150
10247992220,2cyclictest0-21swapper/000:00:560
10247992220,1cyclictest22383-21apt-get20:43:010
1024799220,22cyclictest0-21swapper/021:16:050
1024799220,22cyclictest0-21swapper/000:33:090
1024799220,21cyclictest18558-21sh22:40:090
1024799220,21cyclictest18027-21sh21:23:180
1024799220,21cyclictest11907-21perf20:20:000
1024799220,21cyclictest0-21swapper/023:36:330
1024799220,1cyclictest19829-21apt-get19:30:000
1024799220,1cyclictest17863-21ssh22:05:220
1024799220,1cyclictest0-21swapper/022:22:440
1024799220,1cyclictest0-21swapper/021:47:020
10260992121,0cyclictest19399-21ssh21:25:103
10260992121,0cyclictest0-21swapper/300:05:353
10260992118,2cyclictest17421-21idleruntime-cro19:25:003
1026099210,21cyclictest0-21swapper/323:04:513
1026099210,1cyclictest8920-21ssh22:30:333
1026099210,0cyclictest0-21swapper/323:55:043
1026099210,0cyclictest0-21swapper/321:57:053
10256992120,1cyclictest28-21ksoftirqd/223:30:132
1025699210,20cyclictest29474-21grep20:55:012
1025699210,20cyclictest271rcuc/219:53:012
1025699210,20cyclictest0-21swapper/220:48:022
1025699210,20cyclictest0-21swapper/220:23:022
1025199210,20cyclictest0-21swapper/119:30:001
1025199210,1cyclictest11902-21munin-run20:20:001
1025199210,0cyclictest0-21swapper/121:08:011
1025199210,0cyclictest0-21swapper/120:43:011
10247992120,1cyclictest14781-21apt-get20:28:030
10247992119,1cyclictest25015-21dump-pmu-power19:40:000
10247992119,1cyclictest17446-21munin-run19:25:000
1024799210,21cyclictest0-21swapper/022:15:490
10260992020,0cyclictest7562-21apt-get20:13:023
10260992020,0cyclictest2364-21apt-get21:08:013
10260992018,2cyclictest9958-21apt-get20:18:023
1026099200,0cyclictest2717-21apt-get20:03:013
1026099200,0cyclictest25174-21apt-get20:48:023
10256992020,0cyclictest0-21swapper/219:58:052
10256992018,1cyclictest11870-21cron20:20:002
1025699200,19cyclictest0-21swapper/220:13:022
1025699200,0cyclictest0-21swapper/219:48:012
1025699200,0cyclictest0-21swapper/219:43:022
10251992020,0cyclictest19829-21apt-get19:30:021
10251992020,0cyclictest0-21swapper/120:35:021
10251992020,0cyclictest0-21swapper/120:23:021
10251992018,1cyclictest29843-21grep19:50:011
10251992018,1cyclictest0-21swapper/120:15:001
1025199200,0cyclictest0-21swapper/119:43:021
10247992020,0cyclictest0-21swapper/019:48:000
10247992018,1cyclictest32239-21awk19:55:010
1024799200,0cyclictest0-21swapper/020:13:030
10260991917,1cyclictest15026-21turbostat19:20:003
1026099190,0cyclictest0-21swapper/319:13:003
10251991919,0cyclictest0-21swapper/119:53:011
10251991918,1cyclictest0-21swapper/119:23:011
10247991919,0cyclictest0-21swapper/021:08:010
10247991919,0cyclictest0-21swapper/020:53:020
10247991919,0cyclictest0-21swapper/020:08:040
10260991818,0cyclictest29963-21apt-get20:58:013
10260991818,0cyclictest0-21swapper/320:53:023
10260991818,0cyclictest0-21swapper/319:48:013
10260991817,1cyclictest1852-21cron21:05:003
10256991818,0cyclictest0-21swapper/220:58:012
10256991818,0cyclictest0-21swapper/220:08:042
10256991818,0cyclictest0-21swapper/219:30:022
10251991818,0cyclictest0-21swapper/119:18:011
10247991818,0cyclictest0-21swapper/019:13:000
10247991816,1cyclictest27091-21idleruntime-cro20:50:010
10260991717,0cyclictest20720-21apt-get19:33:023
10260991717,0cyclictest0-21swapper/320:23:023
10256991717,0cyclictest0-21swapper/219:28:032
10251991717,0cyclictest0-21swapper/120:58:011
1025199170,0cyclictest0-21swapper/120:33:011
10247991717,0cyclictest0-21swapper/020:33:010
1024799170,16cyclictest32719-21apt-get19:58:050
10260991616,0cyclictest0-21swapper/320:28:023
10260991616,0cyclictest0-21swapper/319:58:053
10256991616,0cyclictest0-21swapper/221:03:012
10256991616,0cyclictest0-21swapper/220:43:012
10256991615,1cyclictest0-21swapper/219:38:012
10251991616,0cyclictest0-21swapper/119:12:591
10247991616,0cyclictest0-21swapper/020:58:010
1024799161,15cyclictest30824-21kworker/0:219:44:570
10260991514,1cyclictest7086-21basename20:10:003
10256991515,0cyclictest0-21swapper/220:28:022
10256991515,0cyclictest0-21swapper/219:12:592
10260991414,0cyclictest0-21swapper/319:27:033
1026099140,14cyclictest0-21swapper/319:50:043
1026099140,0cyclictest0-21swapper/319:35:393
10256991414,0cyclictest0-21swapper/219:15:322
1025699140,14cyclictest0-21swapper/221:05:132
1025699140,0cyclictest0-21swapper/220:31:152
1025199140,14cyclictest0-21swapper/120:25:521
1025199140,14cyclictest0-21swapper/119:35:141
1025199140,0cyclictest0-21swapper/121:00:581
1025199140,0cyclictest0-21swapper/120:50:071
10247991414,0cyclictest0-21swapper/021:00:290
10247991414,0cyclictest0-21swapper/020:20:200
10247991414,0cyclictest0-21swapper/019:30:020
1024799140,14cyclictest0-21swapper/020:00:080
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional