You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-08-25 - 01:20

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rack7slot7s.osadl.org (updated Mon Aug 24, 2026 12:43:50)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2553929465,9sleep30-21swapper/307:09:023
2554128875,9sleep00-21swapper/007:09:020
2554328466,8sleep20-21swapper/207:09:052
2540528366,13sleep10-21swapper/107:07:141
186422830,0sleep00-21swapper/009:29:060
287502720,0sleep10-21swapper/110:13:511
25713993532,3cyclictest22791-21kworker/3:110:59:233
2571399350,33cyclictest22791-21kworker/3:112:05:133
25713993129,1cyclictest22791-21kworker/3:112:10:333
2571199280,28cyclictest0-21swapper/210:59:372
25713992724,3cyclictest19403-21kworker/3:209:53:233
2571199270,27cyclictest0-21swapper/210:10:082
2570699264,9cyclictest0-21swapper/008:00:140
25713992523,1cyclictest22791-21kworker/3:110:30:533
2570899240,24cyclictest0-21swapper/108:51:381
2570699248,11cyclictest18290-21diskmemload11:50:130
25713992321,1cyclictest22791-21kworker/3:111:13:233
25713992320,2cyclictest22791-21kworker/3:111:38:333
2571399231,21cyclictest19403-21kworker/3:210:26:143
2571199230,22cyclictest0-21swapper/211:41:542
2570699239,6cyclictest5222-21tr10:00:000
25713992220,2cyclictest22791-21kworker/3:110:45:233
25713992220,1cyclictest22791-21kworker/3:112:19:543
25713992220,1cyclictest22791-21kworker/3:111:41:133
2571399221,20cyclictest19403-21kworker/3:209:48:333
2571399221,20cyclictest19403-21kworker/3:209:48:333
25713992120,1cyclictest22791-21kworker/3:112:34:243
2571399210,20cyclictest22791-21kworker/3:111:50:563
2571399210,20cyclictest0-21swapper/309:34:553
2571399210,19cyclictest0-21swapper/312:26:243
2571199210,21cyclictest0-21swapper/210:47:442
2571199210,21cyclictest0-21swapper/209:39:572
2571199210,20cyclictest0-21swapper/210:34:192
25708992119,1cyclictest0-21swapper/108:10:141
2570699210,21cyclictest0-21swapper/010:57:390
25713992019,1cyclictest22791-21kworker/3:111:59:533
25713992019,1cyclictest22791-21kworker/3:111:01:533
2571199200,20cyclictest0-21swapper/209:26:542
2571199200,19cyclictest0-21swapper/209:18:212
2570899200,19cyclictest0-21swapper/109:28:461
2570699200,1cyclictest0-21swapper/012:02:210
25713991918,1cyclictest22791-21kworker/3:111:33:533
2571399191,1cyclictest0-21swapper/309:15:543
2571199199,7cyclictest26635-21idleruntime-cro08:20:002
25711991917,1cyclictest0-21swapper/209:57:222
2571199190,19cyclictest0-21swapper/212:03:412
314242180,0sleep10-21swapper/111:52:001
2571399186,11cyclictest22791-21kworker/3:111:07:253
25713991817,1cyclictest22791-21kworker/3:110:36:533
25713991817,1cyclictest19403-21kworker/3:209:24:433
2571199180,18cyclictest0-21swapper/212:22:552
2571199180,18cyclictest0-21swapper/210:36:232
25708991817,1cyclictest25-21ksoftirqd/111:22:281
25708991814,1cyclictest18290-21diskmemload10:41:231
2570699188,5cyclictest438-21perf07:25:010
25713991716,1cyclictest22791-21kworker/3:111:28:333
25713991715,2cyclictest22791-21kworker/3:111:22:333
25713991715,2cyclictest22791-21kworker/3:110:40:333
25713991715,2cyclictest19403-21kworker/3:210:10:533
25713991715,1cyclictest22791-21kworker/3:111:19:033
2571199170,17cyclictest0-21swapper/211:30:332
2571199170,17cyclictest0-21swapper/211:03:502
2571199170,16cyclictest0-21swapper/212:18:502
2571199170,14cyclictest0-21swapper/211:50:422
2570899172,1cyclictest18290-21diskmemload09:18:251
2570899170,1cyclictest0-21swapper/110:27:441
2570699178,2cyclictest3061-21awk09:20:000
2570699175,6cyclictest7221-21munin-run10:40:000
25706991715,2cyclictest0-21swapper/011:36:480
25713991615,1cyclictest22791-21kworker/3:110:51:443
25713991615,1cyclictest19403-21kworker/3:210:15:333
25713991615,1cyclictest19403-21kworker/3:209:56:533
25713991614,2cyclictest19403-21kworker/3:210:21:233
25713991614,2cyclictest19403-21kworker/3:210:06:033
25713991614,2cyclictest19403-21kworker/3:209:37:033
2571399160,0cyclictest0-21swapper/312:20:363
25711991613,3cyclictest31811-21apache212:06:302
2571199160,16cyclictest0-21swapper/210:01:122
2570899161,1cyclictest141rcu_preempt10:35:291
2570699160,1cyclictest0-21swapper/011:20:030
25711991513,1cyclictest0-21swapper/210:53:252
2571199150,1cyclictest0-21swapper/211:22:272
2571199150,15cyclictest0-21swapper/212:25:362
2570699150,14cyclictest0-21swapper/010:12:120
2570699150,13cyclictest0-21swapper/012:24:100
2570699150,0cyclictest0-21swapper/009:20:560
2571399143,5cyclictest16185-21apt-get09:05:123
2571399142,0cyclictest0-21swapper/312:04:523
25713991411,2cyclictest278882sleep307:13:093
2571399141,0cyclictest0-21swapper/309:29:133
25713991410,3cyclictest12723-21tr07:50:113
25713991410,3cyclictest12723-21tr07:50:113
2571199149,3cyclictest31999-21tr07:20:222
2571199144,1cyclictest141rcu_preempt10:23:522
2571199142,7cyclictest11466-21basename09:25:002
2571199141,1cyclictest28452-21/usr/sbin/munin12:10:222
25711991411,2cyclictest31839-21wc10:15:182
2571199140,2cyclictest0-21swapper/212:30:152
2571199140,14cyclictest0-21swapper/209:13:452
2570899145,1cyclictest0-21swapper/107:15:111
2570899142,12cyclictest18290-21diskmemload11:46:311
2570899140,13cyclictest0-21swapper/111:09:181
25706991414,0cyclictest0-21swapper/012:16:430
25706991410,3cyclictest6054-21ssh12:35:140
2570699140,13cyclictest0-21swapper/011:44:250
2570699140,13cyclictest0-21swapper/009:36:480
25713991310,2cyclictest20839-21sort11:45:243
2571199139,3cyclictest19902-21wc11:25:262
2571199138,3cyclictest27154-21sed07:10:222
2571199135,4cyclictest225-21systemd-journal08:49:592
2571199133,5cyclictest225-21systemd-journal11:39:592
25711991310,2cyclictest615-21grep08:30:202
25711991310,2cyclictest31608-21cron08:30:002
25711991310,2cyclictest29212-21cron09:55:012
25711991310,2cyclictest27126-21wc11:10:262
25711991310,2cyclictest18149-21cut08:00:192
25711991310,2cyclictest13508-21cron09:00:002
25711991310,2cyclictest11478-21head07:45:212
2570899139,3cyclictest32291-21sh10:55:001
2570899139,3cyclictest30444-21cat09:35:231
2570899136,5cyclictest38150irq/131-snd_hda10:55:141
2570899131,4cyclictest0-21swapper/111:36:341
25708991310,3cyclictest38150irq/131-snd_hda11:17:131
25708991310,2cyclictest5914-21wc12:15:221
25708991310,2cyclictest4324-21grep08:40:141
25708991310,2cyclictest32637-21grep10:15:241
25708991310,2cyclictest30731-21tr08:25:221
25708991310,2cyclictest27705-21tr11:30:201
25708991310,2cyclictest2646-21sed08:35:171
25708991310,2cyclictest12586-21ls09:45:001
2570899130,1cyclictest0-21swapper/109:00:131
2570899130,12cyclictest0-21swapper/111:10:131
2570699139,3cyclictest21194-21grep09:30:170
25706991310,2cyclictest27974-21ls12:10:000
25706991310,2cyclictest25617-21grep08:15:190
25706991310,2cyclictest19149-21grep09:10:130
25706991310,2cyclictest15635-21grep10:45:010
25706991310,2cyclictest15091-21basename07:55:130
25706991310,2cyclictest13322-21ssh10:24:030
25706991310,2cyclictest10089-21cat07:45:000
2570699130,13cyclictest0-21swapper/011:00:560
2571399129,2cyclictest5060-21grep08:40:163
2571399129,2cyclictest32674-21tail08:30:173
2571399129,2cyclictest27150-21cat09:15:003
2571399129,2cyclictest2619-21head08:35:173
2571399129,2cyclictest17124-21readlink08:00:003
2571399129,2cyclictest15982-21cat09:05:003
2571399129,2cyclictest11213-21cat08:55:013
2571399127,3cyclictest22425-21tr08:10:153
2571399120,12cyclictest0-21swapper/312:38:013
2571199129,2cyclictest26903-21sh11:50:002
2571199129,2cyclictest23007-21wc10:30:002
2571199129,2cyclictest16951-21sed09:05:162
2571199129,2cyclictest15715-21tail07:55:162
2571199129,2cyclictest14726-21sh07:55:002
2571199129,2cyclictest14726-21sh07:55:002
2571199128,3cyclictest2348-21grep07:25:262
2571199128,3cyclictest23378-21head08:10:232
2571199128,3cyclictest22307-21head09:30:272
2571199128,3cyclictest21919-21sleep10:10:002
2571199128,3cyclictest20800-21wc08:05:202
2571199122,3cyclictest0-21swapper/211:09:002
2571199120,2cyclictest0-21swapper/211:16:162
2571199120,2cyclictest0-21swapper/208:20:142
2571199120,12cyclictest0-21swapper/212:39:082
2570899129,2cyclictest6270-21cut12:35:151
2570899129,2cyclictest6122-21cut07:35:171
2570899129,2cyclictest14776-21grep10:05:171
2570899128,3cyclictest18950-21head11:25:171
2570899128,3cyclictest1643-21perf08:35:001
2570899128,2cyclictest14127-21ls12:20:181
2570899124,7cyclictest6262-21munin-node07:43:521
2570899124,4cyclictest225-21systemd-journal10:00:561
2570899122,4cyclictest0-21swapper/110:47:221
2570899120,12cyclictest0-21swapper/112:13:121
2570699129,2cyclictest766-21sleep11:15:000
2570699129,2cyclictest3259-21wc11:15:260
2570699129,2cyclictest31413-21wc12:30:230
2570699129,2cyclictest21843-21grep12:25:140
2570699129,2cyclictest20275-21taskset10:08:460
2570699129,2cyclictest19856-21head11:45:150
2570699129,2cyclictest18962-21sed11:05:280
2570699129,2cyclictest12146-21head08:55:170
2570699128,3cyclictest20120-21grep11:25:270
2570699128,2cyclictest0-21swapper/009:50:140
2570699120,4cyclictest0-21swapper/010:15:110
2571399118,2cyclictest5778-21cut10:00:153
2571399118,2cyclictest5419-21ssh09:40:173
2571399118,2cyclictest3368-21expr07:30:133
2571399118,2cyclictest32421-21grep07:20:263
2571399118,2cyclictest2329-21unixbench-2d07:25:263
2571399117,3cyclictest25167-21cut08:15:163
2571399110,10cyclictest18490-21sed08:00:213
2571199118,2cyclictest9757-21tail08:50:192
2571199118,2cyclictest7289-21ssh11:56:552
2571199118,2cyclictest5226-21uname08:40:192
2571199118,2cyclictest29029-21tail07:15:152
2571199118,2cyclictest11071-21ssh09:43:482
2571199118,2cyclictest10513-21ssh10:41:022
25711991110,1cyclictest0-21swapper/209:45:152
25711991110,1cyclictest0-21swapper/209:45:152
2571199110,10cyclictest7345-21taskset07:37:032
2570899119,1cyclictest8210-21cat08:45:271
2570899118,2cyclictest9758-21ssh11:58:511
2570899118,2cyclictest9265-21ssh10:20:521
2570899118,2cyclictest26318-21taskset08:16:071
2570899118,2cyclictest26050-21ssh09:14:161
2570899118,2cyclictest2500-21sh12:32:431
2570899118,2cyclictest23321-21chrt10:30:101
2570899118,2cyclictest21303-21cat09:30:191
2570899118,2cyclictest18281-21cron09:10:011
2570899118,2cyclictest15034-21ssh12:01:261
2570899118,2cyclictest13767-21ssh11:41:161
2570899118,2cyclictest13351-21head07:50:151
2570899118,2cyclictest13351-21head07:50:141
2570899118,2cyclictest12864-21ssh09:45:031
2570899118,2cyclictest12864-21ssh09:45:031
2570899118,2cyclictest10055-21head11:00:221
2570899118,2cyclictest0-21swapper/112:25:551
2570899117,3cyclictest24246-21ssh12:07:071
2570899117,3cyclictest0-21swapper/109:20:111
2570899116,4cyclictest13168-21taskset08:55:361
2570899115,2cyclictest225-21systemd-journal07:23:091
25708991110,1cyclictest38150irq/131-snd_hda09:55:201
2570899110,10cyclictest21325-21ssh09:50:111
2570899110,10cyclictest21187-21expr08:05:261
2570699118,2cyclictest8460-21ssh11:57:510
2570699118,2cyclictest7800-21tr08:45:220
2570699118,2cyclictest6467-21cat07:35:190
2570699118,2cyclictest6232-21cat10:00:150
2570699118,2cyclictest5703-21cat09:40:180
2570699118,2cyclictest29808-21wc08:25:150
2570699118,2cyclictest26740-21tr11:30:150
2570699118,2cyclictest25893-21ssh10:30:480
2570699118,2cyclictest21221-21ssh10:28:430
2570699118,2cyclictest17752-21awk10:45:250
2570699118,2cyclictest1499-21tail07:25:170
2570699118,2cyclictest1423-21ssh12:13:020
2570699118,2cyclictest13901-21head09:45:180
2570699118,2cyclictest13901-21head09:45:170
2570699117,3cyclictest5067-21grep08:40:160
2570699114,3cyclictest225-21systemd-journal07:15:000
2571399108,1cyclictest20332-21sed08:05:163
2571399108,1cyclictest12174-21chrt07:47:513
2571399107,2cyclictest8986-21nfs4_client07:40:193
2571399107,2cyclictest27037-21find08:20:013
2571399106,3cyclictest28602-21grep07:15:123
2571399105,2cyclictest225-21systemd-journal08:55:093
2571399100,9cyclictest5454-21cat07:35:123
2571199107,2cyclictest4305-21cut07:30:202
2571199107,2cyclictest2157-21tr08:35:142
2571199100,5cyclictest225-21systemd-journal09:05:002
2570899108,1cyclictest12429-21sed07:50:001
2570899107,2cyclictest4818-21head07:30:241
2570899107,2cyclictest27883-21grep08:20:171
2570899107,2cyclictest18044-21uniq08:00:191
2570899107,2cyclictest15231-21tr07:55:141
2570899107,2cyclictest1017-21cat07:25:151
2570899100,2cyclictest0-21swapper/107:10:141
2570699108,1cyclictest3851-21tail07:30:150
2570699108,1cyclictest17349-21awk09:05:210
2570699107,2cyclictest740-21grep08:30:230
2570699107,2cyclictest29793-21cat07:15:230
2570699107,2cyclictest27371-21grep08:20:140
2570699107,2cyclictest24682-21tr10:50:130
2570699107,2cyclictest20040-21cut08:05:150
2570699107,2cyclictest10652-21expr07:45:160
2570699100,9cyclictest22148-21dpkg08:10:130
2570699100,9cyclictest14250-21unixbench_multi07:50:240
2570699100,9cyclictest14250-21unixbench_multi07:50:230
2570699100,9cyclictest10439-21python08:50:270
257139997,1cyclictest30381-21cat08:25:183
257139996,2cyclictest6815-21grep08:45:153
257119997,1cyclictest8067-21wc07:40:122
257069997,1cyclictest2329-21expr08:35:150
257069996,2cyclictest14032-21cut09:00:140
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional