You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-06-16 - 02:49

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack7slot7s.osadl.org (updated Tue Jun 16, 2026 00:43:51)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
12817210187,9sleep00-21swapper/019:06:010
1298129582,9sleep20-21swapper/219:08:072
1294729363,10sleep30-21swapper/319:07:403
1075728971,13sleep10-21swapper/119:05:011
13214993534,1cyclictest0-21swapper/022:52:020
1321999270,1cyclictest0-21swapper/221:36:202
1321699261,1cyclictest5952-21diskmemload21:30:121
13224992524,1cyclictest0-21swapper/323:33:573
1321999250,25cyclictest0-21swapper/222:25:542
1322499241,22cyclictest0-21swapper/300:28:583
1321699240,24cyclictest0-21swapper/122:25:551
1321699240,24cyclictest0-21swapper/100:25:521
13219992219,3cyclictest32-21ksoftirqd/223:40:002
1321699220,1cyclictest5952-21diskmemload22:57:241
1321499220,1cyclictest0-21swapper/023:42:510
1321499220,0cyclictest0-21swapper/000:30:310
13219992120,1cyclictest0-21swapper/223:26:502
1321999200,18cyclictest0-21swapper/222:30:042
1321699200,2cyclictest0-21swapper/123:59:481
13214992020,0cyclictest0-21swapper/000:12:270
13214992020,0cyclictest0-21swapper/000:12:260
1321999190,1cyclictest0-21swapper/200:04:532
1321999190,15cyclictest0-21swapper/220:55:132
13216991918,1cyclictest0-21swapper/122:47:471
1321699190,18cyclictest0-21swapper/121:48:491
1321499190,12cyclictest0-21swapper/021:42:420
13219991812,2cyclictest32-21ksoftirqd/223:15:182
1321999180,18cyclictest0-21swapper/221:31:432
1321699188,5cyclictest25-21ksoftirqd/122:17:331
1321699180,1cyclictest0-21swapper/122:30:031
13214991817,1cyclictest0-21swapper/023:55:120
13224991717,0cyclictest0-21swapper/322:55:593
13219991714,2cyclictest0-21swapper/223:04:052
13216991715,2cyclictest38150irq/131-snd_hda19:15:171
1321699171,1cyclictest141rcu_preempt23:25:581
1321699171,0cyclictest141rcu_preempt22:41:251
1321699170,15cyclictest0-21swapper/123:12:361
13214991710,6cyclictest8375-21apache223:37:230
183912160,0sleep20-21swapper/200:09:492
1322499165,5cyclictest26411-21runrttasks00:14:333
1322499165,5cyclictest26411-21runrttasks00:14:323
13224991612,2cyclictest17134-21sh21:35:183
1322499160,1cyclictest0-21swapper/322:52:443
1322499160,1cyclictest0-21swapper/300:22:023
1322499160,15cyclictest0-21swapper/321:43:083
13219991614,1cyclictest0-21swapper/223:07:152
1321999160,15cyclictest0-21swapper/200:30:102
1321699168,1cyclictest25-21ksoftirqd/123:41:281
1321699162,0cyclictest141rcu_preempt23:16:111
13216991611,4cyclictest23978-21hwlatdetect19:30:161
1321499160,0cyclictest0-21swapper/000:29:360
13224991514,1cyclictest0-21swapper/322:20:573
13224991512,2cyclictest9221-21apache_processe23:45:123
1322499150,1cyclictest0-21swapper/321:50:353
1322499150,15cyclictest0-21swapper/322:27:223
13219991513,1cyclictest32-21ksoftirqd/222:08:322
13219991512,2cyclictest32-21ksoftirqd/222:15:222
1321999150,1cyclictest0-21swapper/223:40:352
13216991513,1cyclictest25-21ksoftirqd/121:55:001
13216991512,2cyclictest25-21ksoftirqd/120:45:231
13216991511,3cyclictest4164-21wc19:55:251
1321699150,15cyclictest0-21swapper/121:25:521
1321699150,14cyclictest0-21swapper/121:41:381
1321699150,0cyclictest0-21swapper/100:03:321
1321499155,1cyclictest5952-21diskmemload21:37:020
1321499153,7cyclictest24300-21idleruntime-cro23:34:590
1321499151,7cyclictest25315-21perf19:35:000
13214991512,2cyclictest19181-21ssh23:50:220
13214991512,1cyclictest0-21swapper/023:25:370
1321499150,2cyclictest0-21swapper/000:01:470
1321499150,0cyclictest0-21swapper/000:19:370
13224991414,0cyclictest0-21swapper/300:03:343
13224991413,1cyclictest0-21swapper/323:50:113
13224991413,1cyclictest0-21swapper/321:45:343
13224991410,2cyclictest9499-21cat21:30:253
1322499140,12cyclictest32101-21grep21:25:143
13219991412,1cyclictest32-21ksoftirqd/223:30:072
13219991411,2cyclictest19558-21tail23:50:242
13219991411,2cyclictest15503-21sed21:15:152
13219991410,3cyclictest15991-21cut23:10:182
1321999140,14cyclictest0-21swapper/222:36:162
1321999140,13cyclictest27678-21sensors23:55:242
13216991413,1cyclictest0-21swapper/123:37:551
13216991412,1cyclictest25-21ksoftirqd/121:39:521
13216991412,1cyclictest0-21swapper/122:20:271
13216991412,1cyclictest0-21swapper/122:00:081
1321699140,13cyclictest3613-21basename00:20:131
1321699140,13cyclictest0-21swapper/123:02:361
1321499142,1cyclictest0-21swapper/020:20:130
1321499141,7cyclictest17977-21grep19:20:010
1321499140,1cyclictest0-21swapper/023:20:540
13224991310,2cyclictest28673-21grep19:40:143
13224991310,2cyclictest26753-21grep19:35:223
13224991310,2cyclictest25617-21head23:35:183
13224991310,2cyclictest21380-21sh22:35:183
13224991310,2cyclictest20344-21head20:30:173
1322499130,12cyclictest0-21swapper/321:20:363
1321999139,3cyclictest27208-21rm00:15:002
1321999139,3cyclictest27208-21rm00:15:002
1321999132,6cyclictest0-21swapper/222:14:182
13219991311,1cyclictest32-21ksoftirqd/200:21:012
13219991310,2cyclictest5977-21grep21:10:002
13219991310,2cyclictest32-21ksoftirqd/200:35:062
13219991310,2cyclictest26231-21basename22:00:122
13219991310,2cyclictest109372sleep221:50:232
1321999130,12cyclictest18871-21grep20:25:262
1321699139,3cyclictest24466-21grep20:40:131
1321699139,3cyclictest22310-21sed00:30:251
1321699139,3cyclictest17941-21tail20:25:171
1321699138,4cyclictest26378-21ssh21:21:121
1321699137,1cyclictest141rcu_preempt00:18:071
1321699136,1cyclictest25-21ksoftirqd/120:37:051
1321699134,8cyclictest11462-21grep22:10:181
1321699134,5cyclictest16737-21cpuspeed_turbos23:30:161
1321699133,7cyclictest8375-21apache223:22:551
1321699131,2cyclictest0-21swapper/121:57:371
1321699131,11cyclictest0-21swapper/121:16:151
13216991310,2cyclictest2238-21grep21:00:171
1321499132,6cyclictest14581-21munin-run20:20:000
13214991312,1cyclictest0-21swapper/020:45:110
13214991310,2cyclictest3795-21tr22:25:150
13214991310,2cyclictest18868-21sed20:25:270
13214991310,2cyclictest11745-21cat22:10:190
1321499130,1cyclictest0-21swapper/023:17:190
1321499130,12cyclictest11551-21cat20:10:280
1321499130,12cyclictest0-21swapper/022:20:560
1322499129,2cyclictest8666-21sed23:05:293
1322499129,2cyclictest564-21chrt23:59:163
1322499129,2cyclictest28851-21cut22:40:163
1322499129,2cyclictest23798-21tr23:15:143
1322499129,2cyclictest2376-21cron19:54:593
1322499129,2cyclictest19145-21grep21:55:203
1322499129,2cyclictest18665-21turbostat.cron00:10:013
1322499129,2cyclictest16357-21grep23:10:193
1322499129,2cyclictest14590-21awk20:20:003
1322499129,2cyclictest1301-21wc21:00:003
1322499128,3cyclictest16957-21sh20:25:013
1322499128,2cyclictest199362sleep322:15:163
1322499120,1cyclictest0-21swapper/300:39:113
1322499120,11cyclictest0-21swapper/300:30:123
1321999129,2cyclictest6732-21tr21:10:152
1321999129,2cyclictest6594-21sort22:45:272
1321999129,2cyclictest32581-21tail21:25:162
1321999129,2cyclictest29377-21sed22:20:262
1321999129,2cyclictest25660-21grep19:35:142
1321999129,2cyclictest1933-21grep19:50:252
1321999128,3cyclictest22663-21cut20:35:192
1321999128,3cyclictest16973-21awk20:25:012
1321999128,3cyclictest11689-21grep20:10:282
1321999128,2cyclictest32-21ksoftirqd/219:20:012
1321999126,4cyclictest8378-21apache220:51:582
1321999125,5cyclictest291irq_work/221:22:212
1321999124,4cyclictest225-21systemd-journal21:45:352
1321999120,11cyclictest21695-21cron20:35:002
1321699129,2cyclictest7735-21tr21:10:201
1321699129,2cyclictest4171-21cron00:40:001
1321699129,2cyclictest31519-21grep20:55:131
1321699129,2cyclictest20926-21sed00:10:261
1321699129,2cyclictest20926-21sed00:10:251
1321699129,2cyclictest14861-21wc19:10:241
1321699128,3cyclictest25839-21tr19:35:151
1321699128,3cyclictest25-21ksoftirqd/120:15:161
1321699127,4cyclictest1142-21runrttasks22:09:481
1321699125,2cyclictest25-21ksoftirqd/120:35:001
1321699122,10cyclictest5952-21diskmemload22:54:181
1321699121,3cyclictest225-21systemd-journal20:03:571
13216991210,1cyclictest1407-21tr19:50:201
13216991210,1cyclictest0-21swapper/123:51:211
1321499129,2cyclictest32616-21grep23:00:270
1321499129,2cyclictest25729-21grep21:59:590
1321499129,1cyclictest141rcu_preempt23:08:210
1321499122,3cyclictest141rcu_preempt22:38:180
13214991210,1cyclictest12834-21head22:30:160
13214991210,1cyclictest10723-21ls21:50:230
1321499120,9cyclictest0-21swapper/022:49:440
1321499120,6cyclictest2238-21/usr/sbin/munin22:05:120
1321499120,12cyclictest0-21swapper/021:13:290
1321499120,11cyclictest14470-21cat19:10:180
1322499118,2cyclictest8450-21sort20:05:213
1322499118,2cyclictest5980-21ssh22:06:473
1322499118,2cyclictest4990-21ntp_states21:05:203
1322499118,2cyclictest459-21ssh22:04:053
1322499118,2cyclictest29816-21ssh00:15:403
1322499118,2cyclictest29113-21apt20:50:113
1322499118,2cyclictest28639-21chrt20:47:303
1322499118,2cyclictest2842-21ssh23:21:293
1322499118,2cyclictest2645-21open_inodes21:00:213
1322499118,2cyclictest19341-21ls20:30:003
1322499118,2cyclictest14978-21ssh21:15:133
1322499118,2cyclictest12500-21ssh22:30:163
1322499118,2cyclictest10654-21ssh23:25:573
1322499118,2cyclictest10521-21ssh22:10:093
1322499117,3cyclictest7378-21ssh21:10:183
1322499117,2cyclictest5541-21grep22:45:163
1322499110,10cyclictest3574-21ssh23:41:013
1322499110,10cyclictest19524-21tail19:20:193
1322499110,10cyclictest1913-21ssh23:01:443
1321999118,2cyclictest6743-21head20:00:262
1321999118,2cyclictest3226-21taskset21:00:462
1321999118,2cyclictest31102-21cut19:45:172
1321999118,2cyclictest31085-21sh22:40:402
1321999118,2cyclictest28822-21chrt20:49:532
1321999118,2cyclictest2881-21tr19:55:152
1321999118,2cyclictest27581-21cut00:15:132
1321999118,2cyclictest25970-21sed20:40:252
1321999118,2cyclictest25166-21expr21:40:142
1321999118,2cyclictest23408-21tr19:30:132
1321999118,2cyclictest22711-21taskset19:27:312
1321999118,2cyclictest18572-21grep21:55:152
1321999118,2cyclictest15128-21grep22:50:252
1321999118,2cyclictest14831-21sh23:48:142
1321999118,2cyclictest14133-21grep20:15:252
1321999118,2cyclictest1405-21grep23:20:262
1321999118,2cyclictest13651-21wc00:25:252
1321999117,3cyclictest20072-21grep19:20:252
1321999114,1cyclictest32-21ksoftirqd/222:57:322
1321699119,1cyclictest7475-21cut20:05:161
1321699119,1cyclictest21229-21sed22:35:181
1321699118,2cyclictest30068-21munin-plugin-st19:44:591
1321699118,2cyclictest29730-21fschecks_count20:50:161
1321699118,2cyclictest11458-21chrt20:10:251
1321699118,2cyclictest10492-21tr23:45:201
1321699118,2cyclictest10290-21ssh00:05:051
1321699116,1cyclictest11442-21ssh23:07:321
1321499119,1cyclictest15434-21tr23:10:150
1321499118,2cyclictest690-21ssh22:04:130
1321499118,2cyclictest5492-21sed00:20:250
1321499118,2cyclictest4337-21grep19:55:260
1321499118,2cyclictest30538-21cat22:40:270
1321499118,2cyclictest30233-21gpgv23:00:000
1321499118,2cyclictest29851-21cat00:35:180
1321499118,2cyclictest28230-21grep19:40:120
1321499118,2cyclictest24986-21grep20:40:160
1321499118,2cyclictest24076-21munin-run20:40:010
1321499118,2cyclictest18392-21seq19:20:030
1321499118,2cyclictest16887-21ssh00:08:390
1321499118,2cyclictest16135-21cat21:15:210
1321499118,2cyclictest10348-21rm21:30:490
1321499113,3cyclictest225-21systemd-journal20:59:550
1321499112,6cyclictest0-21swapper/020:10:000
1322499108,2cyclictest4065-21switchtime19:55:263
1322499108,1cyclictest5895-21cat20:00:173
1322499108,1cyclictest32207-21chrt19:47:153
1322499108,1cyclictest23279-21gzip20:35:243
1322499108,1cyclictest15830-21chrt19:15:133
1322499107,2cyclictest25302-21cut20:40:183
1322499107,2cyclictest23513-21ls19:30:133
1322499107,2cyclictest20776-21chrt19:25:113
1322499107,2cyclictest14163-21head19:10:173
1322499107,2cyclictest10472-21expr20:10:163
1321999108,1cyclictest8147-21head20:05:192
1321999108,1cyclictest28928-21chrt19:40:162
1321699108,1cyclictest15989-21ntpq20:20:211
1321699107,2cyclictest31980-21grep19:45:261
1321699100,3cyclictest20936-21cut19:25:151
1321499108,2cyclictest16033-21ssh23:49:090
1321499108,1cyclictest362-21ssh21:45:020
1321499108,1cyclictest31520-21chrt21:25:010
1321499108,1cyclictest29830-21head20:50:160
1321499108,1cyclictest23614-21grep21:20:130
1321499108,1cyclictest18862-21ssh22:15:050
1321499107,2cyclictest5275-21tr20:00:150
1321499107,2cyclictest30949-21grep19:45:150
1321499105,2cyclictest225-21systemd-journal21:04:560
1321499100,9cyclictest26412-21mii-tool19:35:170
1321499100,9cyclictest19552-21chrt20:30:030
132199997,1cyclictest13401-21taskset19:10:032
132169997,1cyclictest5100-21processes21:05:211
132169990,1cyclictest141rcu_preempt19:20:131
132149998,1cyclictest21819-21cat19:25:200
132149994,2cyclictest452-21systemd-logind21:05:220
132149990,9cyclictest881-21tune2fs19:50:160
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional