You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-06-24 - 06:49

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack7slot7s.osadl.org (updated Wed Jun 24, 2026 00:43:51)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
482329885,9sleep00-21swapper/019:07:460
465429166,9sleep30-21swapper/319:05:373
480329064,8sleep20-21swapper/219:07:332
493628567,14sleep10-21swapper/119:09:161
508399280,25cyclictest0-21swapper/221:55:022
508399280,1cyclictest0-21swapper/222:25:292
507899289,16cyclictest25-21ksoftirqd/123:32:331
508399230,23cyclictest0-21swapper/223:54:592
5078992312,3cyclictest225-21systemd-journal23:45:011
507499230,22cyclictest0-21swapper/023:58:400
508599220,2cyclictest0-21swapper/322:55:233
5078992215,7cyclictest30524-21diskmemload23:05:271
507899221,20cyclictest0-21swapper/121:29:411
507499220,2cyclictest0-21swapper/020:10:130
507499220,0cyclictest0-21swapper/023:26:000
508399210,1cyclictest0-21swapper/221:50:562
5078992114,6cyclictest0-21swapper/123:35:111
5078992113,7cyclictest0-21swapper/121:34:011
5078992112,8cyclictest0-21swapper/123:20:031
507899210,1cyclictest0-21swapper/120:35:141
507499211,19cyclictest0-21swapper/022:05:120
508599200,1cyclictest0-21swapper/323:30:263
508599200,19cyclictest0-21swapper/323:07:533
508599200,0cyclictest0-21swapper/322:12:553
5083992020,0cyclictest0-21swapper/221:10:242
5083992017,3cyclictest0-21swapper/200:11:052
508399200,20cyclictest0-21swapper/223:02:212
508399200,17cyclictest31332-21apt23:45:012
508399200,0cyclictest0-21swapper/223:48:422
507899207,8cyclictest0-21swapper/122:34:351
507899207,13cyclictest0-21swapper/123:25:311
507899205,5cyclictest25-21ksoftirqd/122:27:031
5078992019,1cyclictest0-21swapper/121:43:461
5078992019,1cyclictest0-21swapper/121:43:451
5078992013,6cyclictest30524-21diskmemload00:16:121
507899200,3cyclictest0-21swapper/122:45:141
507499200,17cyclictest0-21swapper/023:13:380
5083991913,4cyclictest0-21swapper/221:15:222
508399190,19cyclictest0-21swapper/222:43:122
508399190,18cyclictest0-21swapper/200:25:452
507899194,3cyclictest0-21swapper/100:13:431
507899192,16cyclictest30524-21diskmemload22:58:291
5078991915,4cyclictest0-21swapper/121:21:071
5078991912,6cyclictest0-21swapper/122:40:101
5074991911,6cyclictest0-21swapper/021:58:500
508599181,0cyclictest0-21swapper/323:54:313
508599180,0cyclictest0-21swapper/300:02:223
5078991814,3cyclictest22138-21/usr/sbin/munin23:00:181
507499180,17cyclictest0-21swapper/023:45:010
32022180,0sleep00-21swapper/000:05:590
5085991717,0cyclictest0-21swapper/322:32:023
5083991717,0cyclictest0-21swapper/222:18:202
5083991715,1cyclictest0-21swapper/221:30:302
5083991714,3cyclictest32-21ksoftirqd/200:15:142
508399170,15cyclictest0-21swapper/221:20:392
508399170,14cyclictest0-21swapper/221:38:342
507899179,6cyclictest0-21swapper/121:53:061
507899178,9cyclictest0-21swapper/122:50:131
507899177,7cyclictest0-21swapper/122:38:371
507899177,10cyclictest0-21swapper/123:48:281
507899177,10cyclictest0-21swapper/121:39:441
507499172,11cyclictest23870-21apt-get21:25:140
5074991716,1cyclictest0-21swapper/019:30:030
133562170,0sleep20-21swapper/222:35:282
5085991615,1cyclictest12319-21if_enp0s31f620:30:183
508599160,12cyclictest0-21swapper/300:15:143
5083991616,0cyclictest0-21swapper/223:37:412
508399160,1cyclictest0-21swapper/222:01:012
508399160,1cyclictest0-21swapper/221:05:132
507899169,7cyclictest4894-21apache200:07:161
5078991613,3cyclictest25-21ksoftirqd/122:18:041
507499168,2cyclictest0-21swapper/021:14:220
507499160,16cyclictest0-21swapper/023:50:440
508599152,2cyclictest1410-21hddtemp_smartct00:05:223
5085991514,1cyclictest0-21swapper/323:55:143
5085991513,2cyclictest0-21swapper/321:45:143
5085991513,1cyclictest0-21swapper/300:23:233
508399156,1cyclictest0-21swapper/222:10:222
508399152,1cyclictest141rcu_preempt23:08:392
508399150,15cyclictest0-21swapper/200:07:492
508399150,14cyclictest0-21swapper/200:31:562
508399150,13cyclictest0-21swapper/222:52:342
507899159,6cyclictest0-21swapper/121:15:511
507899158,7cyclictest0-21swapper/122:06:541
507899158,6cyclictest0-21swapper/123:19:571
507899154,9cyclictest30524-21diskmemload00:32:331
5078991513,2cyclictest0-21swapper/121:45:231
5078991512,2cyclictest30524-21diskmemload22:10:041
5078991512,1cyclictest0-21swapper/121:05:121
5078991510,4cyclictest21497-21sh23:58:021
507899150,15cyclictest0-21swapper/100:01:141
507499157,4cyclictest23836-21sh22:03:470
5074991511,4cyclictest17308-21kworker/u8:022:26:320
507499150,8cyclictest11227-21perf20:30:000
507499150,5cyclictest0-21swapper/023:00:260
5085991414,0cyclictest0-21swapper/323:19:253
508599141,1cyclictest0-21swapper/321:28:553
5085991411,2cyclictest24398-21sed20:55:183
5085991411,2cyclictest21634-21ssh21:23:403
508599140,0cyclictest0-21swapper/323:26:373
508399141,5cyclictest0-21swapper/221:25:152
5083991413,1cyclictest0-21swapper/222:31:522
508399140,13cyclictest0-21swapper/223:30:592
507899147,7cyclictest0-21swapper/121:10:471
507899147,6cyclictest0-21swapper/100:25:141
5078991410,3cyclictest30524-21diskmemload00:20:081
5078991410,2cyclictest0-21swapper/123:13:171
507899140,3cyclictest0-21swapper/120:55:151
507899140,1cyclictest0-21swapper/100:35:551
507499143,1cyclictest24628-21grep00:00:140
507499140,1cyclictest0-21swapper/021:44:090
507499140,1cyclictest0-21swapper/021:44:080
507499140,14cyclictest0-21swapper/021:15:430
507499140,0cyclictest0-21swapper/023:05:330
508599139,3cyclictest27594-21ssh22:25:173
508599139,3cyclictest20568-21sed20:45:273
508599139,1cyclictest0-21swapper/321:19:363
508599137,2cyclictest0-21swapper/300:37:123
508599131,1cyclictest0-21swapper/322:08:583
5085991311,1cyclictest0-21swapper/321:43:373
5085991311,1cyclictest0-21swapper/321:43:363
5085991310,2cyclictest30160-21sed20:00:263
5085991310,2cyclictest28616-21grep19:55:253
5085991310,2cyclictest11988-21wc22:15:303
5083991311,1cyclictest10092-21apache_volume19:20:122
508399130,13cyclictest0-21swapper/222:08:562
508399130,13cyclictest0-21swapper/200:20:482
508399130,12cyclictest26391-21grep00:00:272
507899138,5cyclictest0-21swapper/123:52:431
507899136,7cyclictest0-21swapper/121:59:371
507899135,8cyclictest30524-21diskmemload22:20:591
507899134,8cyclictest16163-21sendmail_mailst19:30:221
5078991312,1cyclictest38150irq/131-snd_hda22:02:121
5078991312,1cyclictest0-21swapper/121:00:141
507899131,10cyclictest16969-21kworker/1:119:35:141
5078991310,2cyclictest19156-21grep20:45:141
5078991310,2cyclictest13192-21grep19:25:171
507499134,8cyclictest5390-21munin-node00:23:020
5074991311,1cyclictest5805-21wc20:15:230
5074991311,1cyclictest0-21swapper/022:32:560
5074991310,2cyclictest26823-21tail21:45:270
5074991310,2cyclictest25833-21wc19:50:250
5074991310,2cyclictest16799-21ssh21:20:210
5074991310,2cyclictest13440-21cut22:55:190
507499130,12cyclictest20385-21grep00:35:250
507499130,12cyclictest19371-21cron19:40:000
508599129,2cyclictest31798-21grep21:10:193
508599129,2cyclictest2962-21tr00:25:213
508599129,2cyclictest26654-21sort21:00:163
508599129,2cyclictest25357-21ssh22:43:173
508599129,2cyclictest21808-21grep20:50:173
508599129,2cyclictest21638-21chrt22:02:023
508599129,2cyclictest20405-21head19:40:173
508599129,2cyclictest19417-21grep22:20:183
508599129,2cyclictest18664-21cat20:45:003
508599129,2cyclictest17307-21grep19:35:143
508599129,2cyclictest15108-21cut23:35:123
508599128,3cyclictest8420-21head19:15:173
508599128,3cyclictest29008-21sort21:05:163
508599128,3cyclictest11788-21sed19:20:253
508599128,3cyclictest11204-21grep21:55:303
508399129,2cyclictest8346-21grep20:20:362
508399129,2cyclictest28939-21grep22:45:182
508399129,2cyclictest22489-21head20:50:232
5083991211,1cyclictest0-21swapper/223:20:442
5083991210,1cyclictest32063-21grep20:05:132
5083991210,1cyclictest25194-21grep21:45:162
508399120,11cyclictest0-21swapper/223:10:572
507899129,2cyclictest32375-21sort20:05:151
507899129,2cyclictest25186-21head19:50:171
507899129,2cyclictest20331-21tune2fs19:40:171
507899128,3cyclictest10241-21tr20:25:241
507899125,6cyclictest10581-21df_abs19:20:151
5078991211,1cyclictest0-21swapper/120:00:001
5078991210,2cyclictest0-21swapper/119:10:131
507499129,2cyclictest6983-21tr20:20:230
507499129,2cyclictest28783-21cut22:45:170
507499129,2cyclictest24373-21grep20:55:180
507499129,2cyclictest22775-21grep19:45:170
507499129,2cyclictest22775-21grep19:45:160
507499129,2cyclictest15953-21taskset23:35:160
507499129,2cyclictest15669-21head23:15:260
507499129,2cyclictest15563-21grep20:35:270
507499129,2cyclictest10308-21grep22:15:140
507499127,4cyclictest4938-21apache200:11:230
507499124,3cyclictest22426-21ssh23:20:140
507499122,9cyclictest2574-21processes21:50:360
5074991210,1cyclictest3550-21grep00:25:250
5074991210,1cyclictest24253-21grep23:40:160
5074991210,1cyclictest19157-21tr20:45:140
5074991210,1cyclictest17917-21sed20:40:270
507499120,1cyclictest0-21swapper/022:20:460
508599118,2cyclictest8506-21tr21:35:203
508599118,2cyclictest667-21wc20:05:213
508599118,2cyclictest6028-21head19:10:163
508599118,2cyclictest4579-21grep20:15:113
508599118,2cyclictest4536-21grep22:50:163
508599118,2cyclictest27823-21ssh22:45:013
508599118,2cyclictest24719-21ssh23:40:203
508599118,2cyclictest22872-21expr23:20:143
508599118,2cyclictest2112-21ssh21:30:413
508599118,2cyclictest15390-21ssh00:33:013
508599118,2cyclictest14622-21grep20:35:193
508599118,2cyclictest11621-21grep22:35:143
508599118,2cyclictest10761-21wc00:10:243
508599117,3cyclictest1491-21taskset23:45:293
508599114,3cyclictest21015-21/usr/sbin/munin23:00:143
508599113,6cyclictest2312-21missed_timers21:50:333
508599110,1cyclictest0-21swapper/320:25:133
508399119,1cyclictest20546-21grep20:45:262
508399119,1cyclictest19421-21sendmail-msp19:40:002
508399119,1cyclictest17347-21ssh23:55:202
508399118,2cyclictest9798-21cron19:20:012
508399118,2cyclictest30533-21rm23:25:042
508399118,2cyclictest14825-21sed23:15:192
508399115,5cyclictest28126-21kworker/2:021:40:272
508399115,5cyclictest28126-21kworker/2:021:40:262
508399111,1cyclictest0-21swapper/220:10:142
508399110,1cyclictest0-21swapper/200:39:252
507899119,2cyclictest38150irq/131-snd_hda20:00:141
507899115,5cyclictest0-21swapper/120:50:181
507899114,6cyclictest0-21swapper/120:20:261
507899113,4cyclictest225-21systemd-journal20:42:041
507499119,1cyclictest32420-21cut20:05:150
507499119,1cyclictest26542sleep021:31:070
507499119,1cyclictest12740-21wc22:35:220
507499118,2cyclictest6069-21ssh22:12:100
507499118,2cyclictest4555-21tr22:50:150
507499118,2cyclictest31123-21sed20:00:360
507499118,2cyclictest27720-21grep21:00:250
507499118,2cyclictest20536-21ssh22:40:160
507499118,2cyclictest15041-21ssh00:32:460
507499118,2cyclictest13205-21ssh21:37:510
507499118,2cyclictest13180-21ssh23:33:570
507499118,2cyclictest13135-21grep20:30:270
507499117,3cyclictest9305-21sed19:15:260
507499110,2cyclictest0-21swapper/000:15:240
508599108,1cyclictest2213-21chrt20:10:163
508599108,1cyclictest219972sleep319:45:043
508599108,1cyclictest219972sleep319:45:033
508599108,1cyclictest168702sleep319:33:253
508599108,1cyclictest14233-21taskset19:25:453
508599107,2cyclictest7502-21head20:20:273
508599107,2cyclictest5138-21ssh23:10:033
508399108,2cyclictest5649-21sendmail_mailst20:15:192
508399108,2cyclictest28750-21seq19:55:292
508399108,2cyclictest15690-21seq20:36:452
508399108,1cyclictest18434-21ssh22:20:052
508399108,1cyclictest147972sleep219:30:032
508399108,1cyclictest12140-21cut20:30:172
508399107,2cyclictest29479-21cut20:00:152
508399107,2cyclictest26848-21basename21:00:182
508399107,2cyclictest14167-21grep19:25:272
508399107,2cyclictest13969-21sh22:55:222
508399100,3cyclictest225-21systemd-journal20:40:332
507899108,1cyclictest4316-21apache_volume20:15:101
507899108,1cyclictest2040-21cat20:10:141
507899107,2cyclictest22816-21cut19:45:161
507899107,2cyclictest22816-21cut19:45:161
507899107,2cyclictest12188-21expr20:30:171
507899101,8cyclictest0-21swapper/119:15:121
507499108,1cyclictest28423-21cut21:05:140
507499108,1cyclictest26987-21bc19:55:100
507499108,1cyclictest22270-21sed20:50:210
507499108,1cyclictest11482-21sensors_temp19:20:240
507499107,2cyclictest5559-21tr19:10:150
507499107,2cyclictest21647-21chrt19:43:320
50859997,1cyclictest24394-21cat19:50:103
50839998,1cyclictest9088-21basename20:25:112
50839998,1cyclictest6576-21processes19:10:212
50839997,1cyclictest23862-21grep20:55:162
50839996,2cyclictest19903-21expr19:40:152
50839990,5cyclictest25813-21sshd19:50:242
50839990,1cyclictest0-21swapper/219:45:132
50839990,1cyclictest0-21swapper/219:45:122
50749997,1cyclictest13275-21cat19:25:180
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional