You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-03-21 - 00:43

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #5

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackeslot5s.osadl.org (updated Fri Mar 20, 2026 12:44:22)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
22488992523,1cyclictest0-21swapper/310:40:023
2248899250,24cyclictest0-21swapper/309:34:573
22486992523,1cyclictest0-21swapper/211:07:382
22486992520,2cyclictest0-21swapper/210:01:332
2248699250,1cyclictest16601-21ssh10:44:462
2248699250,1cyclictest0-21swapper/212:09:482
22483992522,2cyclictest28069-21sh10:24:161
2248399250,24cyclictest0-21swapper/111:47:001
2248399250,24cyclictest0-21swapper/109:14:111
2248399250,1cyclictest25151-21munin-run09:50:011
2248399250,0cyclictest0-21swapper/110:41:101
22480992525,0cyclictest0-21swapper/010:57:520
2248099250,2cyclictest32230-21diskmemload11:01:010
2248099250,24cyclictest31621-21/usr/sbin/munin10:27:360
2248099250,24cyclictest0-21swapper/012:12:030
2248099250,24cyclictest0-21swapper/011:21:520
2248099250,24cyclictest0-21swapper/009:55:140
22488992422,1cyclictest17015-21diskmemload09:46:263
2248899240,24cyclictest0-21swapper/311:23:553
2248899240,23cyclictest30776-21ssh09:22:453
2248899240,23cyclictest21747-21ssh09:14:293
2248899240,0cyclictest0-21swapper/311:25:103
2248899240,0cyclictest0-21swapper/310:04:453
22486992420,1cyclictest0-21swapper/211:31:042
2248699240,23cyclictest17015-21diskmemload12:23:032
2248699240,23cyclictest17015-21diskmemload11:44:122
2248699240,1cyclictest0-21swapper/209:57:372
2248699240,0cyclictest0-21swapper/211:27:062
22483992422,1cyclictest26696-21ssh11:59:121
22483992421,2cyclictest21291-21rm12:26:051
2248399240,24cyclictest0-21swapper/112:03:471
2248399240,1cyclictest9122-21ssh09:33:521
2248399240,0cyclictest0-21swapper/111:19:411
22480992422,2cyclictest5734-21ssh09:31:060
2248099240,2cyclictest21463-21runrttasks09:14:040
2248099240,23cyclictest0-21swapper/012:37:590
2248099240,23cyclictest0-21swapper/010:17:100
2248099240,1cyclictest0-21swapper/009:21:280
2248099240,0cyclictest17015-21diskmemload11:56:540
22488992323,0cyclictest0-21swapper/312:21:073
22488992323,0cyclictest0-21swapper/312:07:143
2248899230,2cyclictest32263-21awk12:05:003
2248899230,23cyclictest17015-21diskmemload12:36:083
2248899230,23cyclictest0-21swapper/311:52:063
2248899230,23cyclictest0-21swapper/311:17:123
2248899230,23cyclictest0-21swapper/310:50:093
2248899230,23cyclictest0-21swapper/310:32:343
2248899230,22cyclictest19387-21sh09:43:563
2248899230,22cyclictest11975-21sh11:45:013
2248899230,22cyclictest0-21swapper/310:07:293
2248899230,21cyclictest0-21swapper/312:26:003
2248899230,1cyclictest0-21swapper/311:05:433
2248899230,0cyclictest0-21swapper/311:35:493
2248899230,0cyclictest0-21swapper/310:26:433
22486992323,0cyclictest0-21swapper/212:35:282
22486992323,0cyclictest0-21swapper/212:30:282
22486992322,0cyclictest0-21swapper/210:35:592
22486992322,0cyclictest0-21swapper/209:16:162
22486992321,1cyclictest0-21swapper/211:02:292
22486992321,1cyclictest0-21swapper/209:50:422
22486992321,1cyclictest0-21swapper/209:48:452
22486992320,1cyclictest29168-21ntpq10:57:342
2248699230,23cyclictest0-21swapper/210:26:272
2248699230,23cyclictest0-21swapper/210:20:022
2248699230,22cyclictest20295-21grep10:17:292
2248699230,22cyclictest19395-21perf08:10:002
2248699230,22cyclictest17015-21diskmemload11:58:502
2248699230,22cyclictest17015-21diskmemload09:21:032
2248699230,22cyclictest12123-21perf07:55:002
2248699230,0cyclictest0-21swapper/212:10:112
2248699230,0cyclictest0-21swapper/212:03:362
2248699230,0cyclictest0-21swapper/211:48:592
22483992323,0cyclictest17015-21diskmemload09:41:221
22483992323,0cyclictest0-21swapper/111:43:001
22483992323,0cyclictest0-21swapper/111:31:251
22483992323,0cyclictest0-21swapper/111:21:241
22483992323,0cyclictest0-21swapper/111:00:451
22483992323,0cyclictest0-21swapper/109:37:151
22483992323,0cyclictest0-21swapper/109:26:431
22483992322,1cyclictest17015-21diskmemload10:32:141
22483992322,1cyclictest12691-21kworker/u8:111:54:591
22483992321,1cyclictest18174-21rm10:46:561
2248399230,2cyclictest10182-21ssh10:38:221
2248399230,23cyclictest0-21swapper/112:07:041
2248399230,22cyclictest14546-21sendmail-msp08:00:001
2248399230,22cyclictest13070-21sh11:13:271
2248399230,1cyclictest3185-21ssh10:00:161
2248399230,0cyclictest17015-21diskmemload09:16:291
2248399230,0cyclictest0-21swapper/112:39:461
2248399230,0cyclictest0-21swapper/112:22:321
2248399230,0cyclictest0-21swapper/110:27:191
2248399230,0cyclictest0-21swapper/109:52:531
22480992323,0cyclictest0-21swapper/012:15:400
22480992323,0cyclictest0-21swapper/012:05:160
22480992321,2cyclictest0-21swapper/010:35:490
22480992321,2cyclictest0-21swapper/010:00:020
22480992321,1cyclictest22187-21munin-run11:55:000
22480992321,1cyclictest0-21swapper/011:46:080
2248099230,2cyclictest32437-21perf07:30:010
2248099230,23cyclictest0-21swapper/012:26:140
2248099230,22cyclictest6726-21ssh11:07:360
2248099230,22cyclictest30814-21ssh11:32:090
2248099230,22cyclictest28287-21ssh12:01:300
2248099230,22cyclictest25794-21ssh11:27:090
2248099230,22cyclictest24206-21ssh10:20:580
2248099230,22cyclictest20604-21ssh09:45:320
2248099230,22cyclictest16147-21sh10:44:040
2248099230,22cyclictest15698-21ssh12:20:500
2248099230,22cyclictest0-21swapper/009:15:040
2248099230,1cyclictest62550irq/127-eno109:42:280
2248099230,1cyclictest15100-21ssh10:12:270
2248099230,0cyclictest17015-21diskmemload11:16:110
2248099230,0cyclictest0-21swapper/010:45:060
2248099230,0cyclictest0-21swapper/010:31:080
22488992222,0cyclictest0-21swapper/311:32:513
22488992222,0cyclictest0-21swapper/310:35:503
22488992222,0cyclictest0-21swapper/310:20:073
22488992222,0cyclictest0-21swapper/309:50:063
22488992222,0cyclictest0-21swapper/309:36:533
22488992222,0cyclictest0-21swapper/309:15:033
22488992221,1cyclictest0-21swapper/309:55:453
22488992220,1cyclictest28075-21apt-get10:57:293
22488992220,1cyclictest25052-21apt-get08:22:293
22488992220,1cyclictest12778-21ssh11:45:523
22488992220,1cyclictest0-21swapper/312:12:403
2248899220,22cyclictest0-21swapper/311:00:593
2248899220,22cyclictest0-21swapper/310:19:483
2248899220,21cyclictest0-21swapper/311:10:193
2248899220,1cyclictest17015-21diskmemload11:57:293
2248899220,1cyclictest14586-21sh09:05:013
2248899220,1cyclictest0-21swapper/307:47:293
2248899220,0cyclictest17015-21diskmemload12:16:563
2248899220,0cyclictest0-21swapper/310:47:233
2248899220,0cyclictest0-21swapper/310:10:143
22486992222,0cyclictest17015-21diskmemload10:05:462
22486992222,0cyclictest0-21swapper/210:50:192
22486992222,0cyclictest0-21swapper/210:45:562
22486992222,0cyclictest0-21swapper/209:40:182
22486992222,0cyclictest0-21swapper/209:10:252
22486992220,1cyclictest0-21swapper/212:15:442
2248699220,22cyclictest0-21swapper/209:25:202
2248699220,21cyclictest32415-21sh07:30:012
2248699220,21cyclictest12639-21apt-get09:02:302
2248699220,21cyclictest0-21swapper/212:25:232
2248699220,21cyclictest0-21swapper/211:20:302
2248699220,21cyclictest0-21swapper/211:17:342
2248699220,21cyclictest0-21swapper/211:12:152
2248699220,21cyclictest0-21swapper/210:11:232
2248699220,21cyclictest0-21swapper/209:39:292
2248699220,21cyclictest0-21swapper/208:22:292
2248699220,1cyclictest5334-21sh09:30:302
2248699220,1cyclictest3220-21ssh11:36:452
2248699220,1cyclictest0-21swapper/208:32:282
22483992222,0cyclictest17015-21diskmemload10:15:511
22483992222,0cyclictest0-21swapper/112:11:191
22483992222,0cyclictest0-21swapper/111:35:181
22483992222,0cyclictest0-21swapper/111:25:301
22483992222,0cyclictest0-21swapper/110:55:071
22483992222,0cyclictest0-21swapper/109:56:351
22483992221,1cyclictest29396-21apt-get08:30:021
2248399220,22cyclictest21967-21ssh10:50:501
2248399220,21cyclictest27445-21apt-get08:27:291
2248399220,21cyclictest2172-21perf08:40:001
2248399220,21cyclictest17206-21sh10:13:091
2248399220,21cyclictest0-21swapper/112:16:511
2248399220,1cyclictest9988-21apt-get10:07:291
2248399220,1cyclictest28944-21apt-get09:22:291
2248399220,1cyclictest26808-21ssh12:31:531
2248399220,0cyclictest0-21swapper/111:07:161
22480992222,0cyclictest0-21swapper/011:11:050
22480992221,1cyclictest424-21apt-get07:32:290
22480992221,1cyclictest424-21apt-get07:32:290
22480992221,1cyclictest15045-21apt-get08:02:290
2248099220,22cyclictest0-21swapper/010:05:290
2248099220,22cyclictest0-21swapper/009:27:190
2248099220,21cyclictest30236-21apt-get08:32:280
2248099220,21cyclictest29381-21sh08:30:010
2248099220,21cyclictest25741-21ssh09:50:430
2248099220,21cyclictest17015-21diskmemload10:52:350
2248099220,21cyclictest0-21swapper/011:35:540
2248099220,1cyclictest0-21swapper/009:35:150
22488992119,1cyclictest9386-21awk08:55:003
22488992119,1cyclictest2640-21apt-get08:42:303
2248899210,20cyclictest0-21swapper/307:12:273
2248899210,0cyclictest17015-21diskmemload12:30:233
2248899210,0cyclictest0-21swapper/307:52:293
22486992119,1cyclictest17007-21munin-run08:05:002
2248699210,21cyclictest0-21swapper/211:51:112
2248699210,20cyclictest4952-21sendmail-msp07:40:012
2248699210,20cyclictest32633-21apt-get08:37:282
2248699210,20cyclictest0-21swapper/208:47:292
2248699210,20cyclictest0-21swapper/208:12:292
2248699210,1cyclictest28092-21apt-get07:22:282
2248699210,1cyclictest1440-21ssh10:30:152
2248699210,0cyclictest0-21swapper/207:47:292
22483992120,0cyclictest0-21swapper/107:52:291
22483992118,2cyclictest2418-21ls07:35:011
22483992118,2cyclictest2418-21ls07:35:011
22480992120,1cyclictest9391-21munin-run08:55:000
2248099210,21cyclictest0-21swapper/012:30:020
2248099210,20cyclictest0-21swapper/008:17:280
2248099210,20cyclictest0-21swapper/007:22:280
2248099210,0cyclictest0-21swapper/007:42:270
22488992020,0cyclictest0-21swapper/309:07:283
22488992020,0cyclictest0-21swapper/308:17:293
22488992018,1cyclictest622-21gdbus07:45:013
2248899200,0cyclictest0-21swapper/307:32:293
2248899200,0cyclictest0-21swapper/307:32:293
2248899200,0cyclictest0-21swapper/307:17:283
22486992019,1cyclictest5388-21apt-get07:42:272
2248699200,0cyclictest0-21swapper/208:55:022
22483992020,0cyclictest27239-21apt-get07:20:021
22483992020,0cyclictest0-21swapper/108:17:281
22483992020,0cyclictest0-21swapper/107:12:271
22483992019,0cyclictest0-21swapper/107:27:281
2248399200,0cyclictest0-21swapper/108:55:021
22480992020,0cyclictest0-21swapper/008:47:290
22480992020,0cyclictest0-21swapper/008:20:020
22480992019,1cyclictest10178-21apt-get07:52:290
22480992019,1cyclictest0-21swapper/008:42:300
22480992017,2cyclictest14601-21munin-run09:05:010
2248099200,0cyclictest0-21swapper/008:12:290
2248099200,0cyclictest0-21swapper/008:07:290
22488991919,0cyclictest0-21swapper/308:57:313
22483991919,0cyclictest15083-21apt-get09:07:281
22483991919,0cyclictest0-21swapper/107:42:271
22483991918,1cyclictest19835-21apt-get08:12:291
22483991917,1cyclictest17003-21perf08:05:001
22480991919,0cyclictest0-21swapper/007:57:280
22480991919,0cyclictest0-21swapper/007:37:100
22488991818,0cyclictest0-21swapper/308:32:283
22488991818,0cyclictest0-21swapper/307:37:303
22488991817,1cyclictest12574-21apt-get07:57:283
22488991817,0cyclictest0-21swapper/307:20:023
22486991817,1cyclictest22233-21apt-get08:17:282
22486991817,0cyclictest0-21swapper/209:07:282
2248699180,17cyclictest0-21swapper/207:17:282
22483991818,0cyclictest0-21swapper/108:07:291
22483991816,1cyclictest5036-21apt-get08:47:291
22480991818,0cyclictest22884-21apt-get07:12:280
2248899170,1cyclictest0-21swapper/308:37:283
2248899170,16cyclictest0-21swapper/308:12:293
2248699170,16cyclictest14516-21sh08:00:002
22483991716,1cyclictest24191-21apt-get08:20:021
22480991715,1cyclictest27244-21latency_hist07:20:000
22486991616,0cyclictest0-21swapper/208:42:302
22486991616,0cyclictest0-21swapper/207:12:272
2248899150,1cyclictest0-21swapper/308:07:343
2248699150,14cyclictest1103-21hddtemp_smartct07:32:312
2248699150,14cyclictest1103-21hddtemp_smartct07:32:312
22483991515,0cyclictest0-21swapper/108:42:301
22480991515,0cyclictest0-21swapper/008:37:280
22480991515,0cyclictest0-21swapper/007:47:290
2248099150,1cyclictest10220-21apt-get08:57:300
22488991414,0cyclictest0-21swapper/308:25:293
2248899140,14cyclictest0-21swapper/308:00:013
2248899140,14cyclictest0-21swapper/307:25:133
2248899140,0cyclictest0-21swapper/308:45:313
2248699140,14cyclictest0-21swapper/208:25:132
2248699140,0cyclictest0-21swapper/208:51:042
22483991414,0cyclictest0-21swapper/109:00:111
22483991414,0cyclictest0-21swapper/107:45:281
22483991414,0cyclictest0-21swapper/107:15:501
2248399140,14cyclictest0-21swapper/107:35:301
2248099140,14cyclictest0-21swapper/009:05:120
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional