You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-26 - 02:39

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #5

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rackeslot5s.osadl.org (updated Sat Apr 25, 2026 12:44:21)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2401599260,25cyclictest3475-21ssh11:31:430
2402999250,24cyclictest0-21swapper/310:48:193
24026992522,2cyclictest26137-21sh11:23:082
2402699251,23cyclictest0-21swapper/212:35:312
2402699250,24cyclictest0-21swapper/208:25:312
2401999250,24cyclictest28861-21pmu-power09:50:391
2401999250,24cyclictest13431-21ssh10:39:491
2401999250,24cyclictest0-21swapper/111:26:291
24015992522,2cyclictest16913-21perf09:40:010
2401599250,24cyclictest0-21swapper/009:27:250
2401599250,1cyclictest14450-21ssh10:08:430
24029992423,1cyclictest18149-21diskmemload12:06:473
2402999240,24cyclictest0-21swapper/311:19:563
2402999240,23cyclictest31295-21cron08:30:003
2402999240,1cyclictest0-21swapper/310:39:493
2402699240,23cyclictest0-21swapper/211:01:302
2402699240,1cyclictest24257-21ssh11:52:142
2402699240,1cyclictest0-21swapper/212:02:542
24019992423,1cyclictest18149-21diskmemload11:39:521
24019992423,1cyclictest0-21swapper/111:00:371
2401999242,1cyclictest11931-21head10:05:371
2401999240,24cyclictest0-21swapper/111:24:331
2401999240,23cyclictest27127-21awk09:50:011
2401999240,23cyclictest0-21swapper/112:27:261
2401999240,23cyclictest0-21swapper/110:02:351
2401999240,1cyclictest31299-21cron08:30:001
2401999240,0cyclictest0-21swapper/111:44:381
2401999240,0cyclictest0-21swapper/110:48:191
24015992423,1cyclictest0-21swapper/011:52:180
2401599240,23cyclictest30094-21ssh09:51:370
2401599240,23cyclictest18149-21diskmemload09:17:400
2401599240,0cyclictest18149-21diskmemload10:11:250
24029992323,0cyclictest30571-21ssh11:26:293
24029992323,0cyclictest18149-21diskmemload11:24:333
24029992323,0cyclictest18149-21diskmemload11:11:183
24029992323,0cyclictest0-21swapper/311:55:203
24029992323,0cyclictest0-21swapper/309:35:463
24029992320,2cyclictest6371-21sh10:31:423
2402999230,2cyclictest18318-21ssh11:45:533
2402999230,23cyclictest19085-21ssh12:19:463
2402999230,23cyclictest18149-21diskmemload12:12:093
2402999230,23cyclictest15551-21ssh11:44:383
2402999230,23cyclictest0-21swapper/311:06:003
2402999230,23cyclictest0-21swapper/310:58:283
2402999230,23cyclictest0-21swapper/310:41:413
2402999230,23cyclictest0-21swapper/309:47:323
2402999230,23cyclictest0-21swapper/309:24:313
2402999230,22cyclictest3185-21sh10:29:483
2402999230,22cyclictest0-21swapper/310:11:393
2402999230,0cyclictest18149-21diskmemload09:43:363
2402999230,0cyclictest0-21swapper/312:25:303
24026992323,0cyclictest18149-21diskmemload12:22:092
24026992323,0cyclictest0-21swapper/210:18:242
24026992323,0cyclictest0-21swapper/209:49:402
24026992323,0cyclictest0-21swapper/209:16:562
24026992322,1cyclictest23803-21sh10:49:562
24026992321,2cyclictest18149-21diskmemload11:30:322
2402699231,1cyclictest3550-21idleruntime-cro07:35:002
2402699230,23cyclictest0-21swapper/210:06:482
2402699230,23cyclictest0-21swapper/209:39:042
2402699230,22cyclictest18149-21diskmemload12:31:422
2402699230,22cyclictest0-21swapper/210:50:072
2402699230,0cyclictest0-21swapper/211:11:582
2402699230,0cyclictest0-21swapper/210:24:202
2402699230,0cyclictest0-21swapper/209:21:222
24019992323,0cyclictest0-21swapper/110:19:351
24019992322,1cyclictest18149-21diskmemload12:06:261
24019992321,1cyclictest0-21swapper/111:15:341
24019992321,1cyclictest0-21swapper/111:11:391
24019992321,1cyclictest0-21swapper/110:58:171
24019992321,1cyclictest0-21swapper/110:20:401
24019992321,1cyclictest0-21swapper/109:28:121
24019992320,2cyclictest3664-21sh12:35:331
2401999231,1cyclictest0-21swapper/107:25:301
2401999230,23cyclictest0-21swapper/112:16:411
2401999230,23cyclictest0-21swapper/111:55:201
2401999230,23cyclictest0-21swapper/110:10:421
2401999230,23cyclictest0-21swapper/109:35:461
2401999230,23cyclictest0-21swapper/109:30:551
2401999230,22cyclictest774-21runrttasks09:18:551
2401999230,22cyclictest3714-21basename08:40:001
2401999230,22cyclictest0-21swapper/111:05:271
2401999230,22cyclictest0-21swapper/109:20:511
2401999230,0cyclictest0-21swapper/112:33:291
2401999230,0cyclictest0-21swapper/112:12:091
24015992323,0cyclictest18149-21diskmemload12:19:030
24015992323,0cyclictest18149-21diskmemload10:40:340
24015992323,0cyclictest0-21swapper/011:46:320
24015992323,0cyclictest0-21swapper/011:40:060
24015992323,0cyclictest0-21swapper/010:45:240
24015992323,0cyclictest0-21swapper/009:58:150
24015992321,2cyclictest18149-21diskmemload10:38:310
24015992321,1cyclictest10970-21perf08:55:010
2401599231,21cyclictest31715-21ssh10:56:210
2401599230,23cyclictest18149-21diskmemload11:19:220
2401599230,23cyclictest0-21swapper/012:38:270
2401599230,23cyclictest0-21swapper/009:48:170
2401599230,22cyclictest4530-21ssh10:30:340
2401599230,22cyclictest22856-21ssh10:15:510
2401599230,22cyclictest22747-21ssh12:22:230
2401599230,22cyclictest0-21swapper/009:10:270
2401599230,21cyclictest32290-21ntp_states10:25:370
24029992222,0cyclictest18149-21diskmemload11:00:373
24029992222,0cyclictest0-21swapper/312:22:063
24029992222,0cyclictest0-21swapper/310:50:183
24029992222,0cyclictest0-21swapper/309:30:553
24029992222,0cyclictest0-21swapper/309:26:183
24029992221,1cyclictest23798-21apt-get09:15:303
24029992220,1cyclictest0-21swapper/311:53:213
24029992220,1cyclictest0-21swapper/310:23:353
2402999220,2cyclictest18419-21ssh09:10:023
2402999220,22cyclictest0-21swapper/312:02:173
2402999220,21cyclictest8512-21perf08:50:013
2402999220,21cyclictest31477-21apt-get07:25:303
2402999220,21cyclictest22020-21ntp_states10:15:373
2402999220,21cyclictest18149-21diskmemload11:35:043
2402999220,1cyclictest7871-21ssh10:01:253
2402999220,1cyclictest0-21swapper/310:05:373
24026992222,0cyclictest0-21swapper/210:02:022
24026992222,0cyclictest0-21swapper/209:10:032
24026992220,2cyclictest5862-21sh11:35:122
24026992220,1cyclictest18433-21chrt11:46:042
24026992220,1cyclictest0-21swapper/210:40:262
2402699220,21cyclictest18149-21diskmemload10:25:122
2402699220,21cyclictest18149-21diskmemload09:50:422
2402699220,21cyclictest1589-21apt-get08:35:312
2402699220,21cyclictest0-21swapper/211:41:542
2402699220,21cyclictest0-21swapper/211:15:332
2402699220,21cyclictest0-21swapper/211:05:212
2402699220,21cyclictest0-21swapper/210:36:192
2402699220,21cyclictest0-21swapper/210:11:252
2402699220,21cyclictest0-21swapper/209:55:072
2402699220,1cyclictest6091-21munin-run07:40:002
2402699220,1cyclictest10070-21df_inode12:10:342
2402699220,1cyclictest0-21swapper/212:15:082
2402699220,1cyclictest0-21swapper/210:55:552
2402699220,1cyclictest0-21swapper/210:30:322
2402699220,0cyclictest18149-21diskmemload09:32:222
2402699220,0cyclictest0-21swapper/212:25:042
2402699220,0cyclictest0-21swapper/211:57:122
24019992222,0cyclictest0-21swapper/112:20:011
24019992222,0cyclictest0-21swapper/112:02:491
24019992222,0cyclictest0-21swapper/111:45:211
24019992222,0cyclictest0-21swapper/111:32:131
24019992222,0cyclictest0-21swapper/109:42:201
24019992220,1cyclictest0-21swapper/111:50:091
24019992220,1cyclictest0-21swapper/110:28:591
2401999220,22cyclictest8517-21wc08:50:011
2401999220,22cyclictest0-21swapper/110:50:181
2401999220,22cyclictest0-21swapper/110:30:441
2401999220,21cyclictest0-21swapper/109:10:071
2401999220,21cyclictest0-21swapper/107:40:021
2401999220,0cyclictest18149-21diskmemload10:42:021
24015992222,0cyclictest0-21swapper/011:20:360
24015992222,0cyclictest0-21swapper/010:22:200
24015992221,1cyclictest18149-21diskmemload11:36:050
24015992221,1cyclictest17219-21taskset08:00:320
24015992221,0cyclictest0-21swapper/007:15:310
24015992220,1cyclictest29295-21apt-get09:20:320
2401599220,2cyclictest7593-21sh10:01:000
2401599220,22cyclictest0-21swapper/011:08:450
2401599220,21cyclictest9540-21apt-get07:45:020
2401599220,21cyclictest9414-21rm12:10:160
2401599220,21cyclictest0-21swapper/010:52:440
2401599220,21cyclictest0-21swapper/008:05:310
2401599220,1cyclictest18149-21diskmemload11:26:340
2401599220,1cyclictest18149-21diskmemload11:00:490
2401599220,1cyclictest0-21swapper/011:55:320
2401599220,0cyclictest18149-21diskmemload09:30:390
2401599220,0cyclictest0-21swapper/012:32:230
24029992120,1cyclictest28918-21cron08:25:003
24029992120,1cyclictest24271-21apt-get07:10:313
24029992120,1cyclictest18149-21diskmemload09:50:213
2402999210,20cyclictest18149-21diskmemload11:30:393
2402999210,20cyclictest13669-21apt-get09:00:323
2402999210,20cyclictest0-21swapper/307:40:323
2402999210,1cyclictest3718-21sendmail08:40:003
2402999210,1cyclictest0-21swapper/309:05:313
2402999210,0cyclictest0-21swapper/312:31:223
2402999210,0cyclictest0-21swapper/309:56:463
24026992120,0cyclictest6775-21apt-get07:40:322
24026992119,1cyclictest26366-21turbostat.cron07:15:012
2402699210,21cyclictest0-21swapper/209:25:042
2402699210,20cyclictest5894-21mailstats12:05:392
2402699210,20cyclictest0-21swapper/209:00:332
2402699210,20cyclictest0-21swapper/208:15:332
2402699210,1cyclictest17199-21ssh09:40:052
2402699210,1cyclictest0-21swapper/209:05:312
2402699210,0cyclictest0-21swapper/208:45:312
2402699210,0cyclictest0-21swapper/208:00:322
24019992121,0cyclictest0-21swapper/107:10:311
24019992119,1cyclictest0-21swapper/109:55:041
2401999210,20cyclictest0-21swapper/107:35:301
24015992121,0cyclictest18149-21diskmemload09:41:410
24015992121,0cyclictest0-21swapper/012:00:230
24015992119,1cyclictest0-21swapper/007:25:010
2401599210,21cyclictest0-21swapper/008:10:310
2401599210,1cyclictest0-21swapper/008:45:310
2401599210,0cyclictest0-21swapper/011:10:010
2401599210,0cyclictest0-21swapper/007:40:020
24029992020,0cyclictest0-21swapper/307:20:313
2402999200,20cyclictest0-21swapper/307:50:323
2402999200,0cyclictest0-21swapper/308:15:323
24026992020,0cyclictest0-21swapper/207:15:322
24026992019,1cyclictest8804-21apt-get08:50:322
24026992019,1cyclictest11251-21apt-get08:55:312
2402699200,20cyclictest0-21swapper/208:30:312
24019992020,0cyclictest0-21swapper/107:50:021
24019992019,1cyclictest4016-21apt-get08:40:321
2401999200,20cyclictest0-21swapper/109:00:321
2401599200,0cyclictest0-21swapper/009:05:310
2401599200,0cyclictest0-21swapper/008:30:310
2401599200,0cyclictest0-21swapper/007:15:010
24026991919,0cyclictest0-21swapper/208:20:322
24019991919,0cyclictest0-21swapper/108:05:311
24019991919,0cyclictest0-21swapper/107:30:321
24019991917,1cyclictest0-21swapper/107:20:011
24015991919,0cyclictest24406-21apt-get08:15:320
24015991919,0cyclictest0-21swapper/009:00:320
24015991919,0cyclictest0-21swapper/008:35:310
24029991818,0cyclictest0-21swapper/308:05:313
24029991817,1cyclictest22004-21apt-get08:10:313
24029991816,1cyclictest28761-21turbostat.cron07:20:013
24019991818,0cyclictest0-21swapper/109:05:311
24019991818,0cyclictest0-21swapper/108:20:321
24019991818,0cyclictest0-21swapper/108:10:311
24019991818,0cyclictest0-21swapper/107:45:311
24015991818,0cyclictest0-21swapper/008:40:320
24015991818,0cyclictest0-21swapper/007:35:300
24015991818,0cyclictest0-21swapper/007:25:310
24029991717,0cyclictest0-21swapper/308:40:323
24029991717,0cyclictest0-21swapper/307:30:323
24029991716,1cyclictest17215-21apt-get08:00:323
24029991715,1cyclictest31615-21apt-get08:30:313
24026991717,0cyclictest0-21swapper/208:10:312
24026991715,1cyclictest0-21swapper/207:50:322
24019991717,0cyclictest0-21swapper/108:00:321
24019991714,2cyclictest13351-21sh09:00:011
24015991716,1cyclictest11702-21apt-get07:50:020
24029991616,0cyclictest0-21swapper/307:45:313
24026991616,0cyclictest0-21swapper/207:55:322
24026991615,1cyclictest0-21swapper/207:25:312
24019991616,0cyclictest0-21swapper/108:50:321
24015991616,0cyclictest0-21swapper/009:00:010
24015991616,0cyclictest0-21swapper/008:25:310
24015991616,0cyclictest0-21swapper/007:55:320
24029991513,2cyclictest11014-21latency_hist08:55:013
24029991513,1cyclictest0-21swapper/309:00:013
24019991514,1cyclictest211rcuc/107:25:011
24029991414,0cyclictest0-21swapper/307:35:203
2402999140,14cyclictest0-21swapper/307:57:113
2402699140,14cyclictest0-21swapper/208:41:122
2402699140,14cyclictest0-21swapper/208:05:072
2402699140,14cyclictest0-21swapper/207:21:502
2402699140,0cyclictest0-21swapper/207:47:432
24019991414,0cyclictest0-21swapper/108:15:131
2401999140,14cyclictest0-21swapper/107:55:331
2401599140,14cyclictest0-21swapper/008:20:300
2401599140,14cyclictest0-21swapper/007:30:020
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional