You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2025-12-08 - 16:11

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #3, slot #6

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rack3slot6s.osadl.org (updated Mon Dec 08, 2025 12:43:49)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
26296212096,18sleep20-21swapper/207:05:282
26442211796,15sleep30-21swapper/307:07:213
26604211695,16sleep00-21swapper/007:09:270
26646211396,12sleep10-21swapper/107:09:561
179712800,1sleep00-21swapper/009:50:590
115322540,0sleep00-21swapper/009:45:120
26679992320,2cyclictest1742-21kworker/u8:209:36:062
2667999230,18cyclictest0-21swapper/211:51:052
26679992219,2cyclictest6293-21kworker/u8:107:59:052
26679992219,2cyclictest6293-21kworker/u8:107:59:042
26679992219,2cyclictest6293-21kworker/u8:107:42:012
26679992219,2cyclictest32674-21kworker/u8:009:09:042
26679992219,2cyclictest28469-21kworker/u8:307:35:052
26679992219,2cyclictest28469-21kworker/u8:307:31:042
26679992219,2cyclictest22556-21kworker/u8:008:16:052
26679992219,2cyclictest19899-21kworker/u8:212:18:062
26679992219,2cyclictest18072-21kworker/u8:209:52:062
26679992219,2cyclictest12756-21kworker/u8:008:03:052
26679992219,2cyclictest12756-21kworker/u8:008:03:042
26679992219,2cyclictest11695-21kworker/u8:210:32:062
26679992218,2cyclictest11629-21kworker/u8:009:55:042
2667999220,17cyclictest0-21swapper/211:46:012
2667999220,17cyclictest0-21swapper/208:57:042
2667799220,3cyclictest27907-21latency08:30:190
2667799220,2cyclictest0-21swapper/010:38:250
2667799220,20cyclictest28990-21ssh11:47:320
2667799220,13cyclictest10547-21taskset09:44:040
26680992118,2cyclictest386-21kworker/u8:010:14:043
26680992118,2cyclictest32674-21kworker/u8:009:10:053
2667999212,2cyclictest0-21swapper/210:48:042
26679992118,2cyclictest6293-21kworker/u8:108:39:052
26679992118,2cyclictest6293-21kworker/u8:108:25:052
26679992118,2cyclictest6268-21kworker/u8:110:16:052
26679992118,2cyclictest4311-21kworker/u8:109:04:052
26679992118,2cyclictest32674-21kworker/u8:008:49:042
26679992118,2cyclictest32674-21kworker/u8:008:44:052
26679992118,2cyclictest28469-21kworker/u8:307:29:052
26679992118,2cyclictest26750-21kworker/u8:007:20:052
26679992118,2cyclictest23581-21kworker/u8:209:26:052
26679992118,2cyclictest19899-21kworker/u8:212:22:052
26679992118,2cyclictest18310-21kworker/u8:011:02:052
26679992118,2cyclictest18256-21kworker/u8:109:22:042
26679992118,2cyclictest18072-21kworker/u8:210:01:052
26679992118,2cyclictest1742-21kworker/u8:209:43:052
26679992118,2cyclictest15928-21kworker/u8:111:42:052
2667999210,4cyclictest31-21ksoftirqd/211:37:042
2667999210,17cyclictest0-21swapper/209:16:022
2667999210,16cyclictest0-21swapper/212:09:052
26678992118,2cyclictest2277-21kworker/u8:112:33:051
26678992118,2cyclictest19899-21kworker/u8:212:24:051
2667799210,3cyclictest0-21swapper/008:35:100
26680992017,2cyclictest32674-21kworker/u8:009:33:043
2668099200,18cyclictest0-21swapper/312:33:053
2667999202,2cyclictest0-21swapper/210:53:052
26679992017,2cyclictest9654-21kworker/u8:211:27:042
26679992017,2cyclictest6836-21kworker/u8:111:59:052
26679992017,2cyclictest28294-21kworker/u8:109:47:042
26679992017,2cyclictest26445-21kworker/u8:012:26:042
2667999201,2cyclictest0-21swapper/212:04:042
2667999201,2cyclictest0-21swapper/210:27:052
2667999200,18cyclictest0-21swapper/210:55:052
2667999200,15cyclictest0-21swapper/208:11:052
2667799200,2cyclictest0-21swapper/011:28:170
26680991916,2cyclictest15928-21kworker/u8:111:45:043
2667999192,9cyclictest0-21swapper/211:08:052
2667999191,2cyclictest0-21swapper/207:19:042
26677991910,7cyclictest25992-21ssh09:25:200
2667799190,2cyclictest0-21swapper/011:32:520
2667799190,17cyclictest10777-21diskmemload12:00:220
2667799190,11cyclictest0-21swapper/010:46:210
2667799190,11cyclictest0-21swapper/009:18:260
112202190,0sleep30-21swapper/311:28:543
26680991815,2cyclictest26445-21kworker/u8:012:23:043
26680991815,2cyclictest11695-21kworker/u8:210:32:013
26680991815,2cyclictest11629-21kworker/u8:009:46:013
26679991815,2cyclictest20276-21kworker/u8:107:10:062
2667999181,2cyclictest0-21swapper/211:10:012
2667999180,2cyclictest0-21swapper/212:38:042
2667999180,2cyclictest0-21swapper/209:34:042
2667999180,2cyclictest0-21swapper/209:11:042
26678991815,2cyclictest15928-21kworker/u8:111:48:051
2667799187,2cyclictest0-21swapper/009:21:570
2667799180,2cyclictest0-21swapper/012:34:240
2667799180,2cyclictest0-21swapper/011:50:280
2667799180,2cyclictest0-21swapper/010:10:300
2667799180,1cyclictest0-21swapper/012:08:390
2667799180,16cyclictest0-21swapper/011:08:540
2667799180,11cyclictest10777-21diskmemload11:38:010
2667799180,11cyclictest0-21swapper/012:20:570
309992170,0sleep30-21swapper/310:05:203
26680991714,2cyclictest9654-21kworker/u8:211:33:053
26680991714,2cyclictest6268-21kworker/u8:110:47:043
26680991714,2cyclictest386-21kworker/u8:010:18:053
26680991714,2cyclictest16736-21kworker/u8:208:07:043
26680991714,2cyclictest12632-21kworker/u8:012:07:053
26680991714,2cyclictest11695-21kworker/u8:210:26:053
26680991714,2cyclictest11283-21kworker/u8:209:16:063
26680991714,2cyclictest0-21swapper/312:27:483
2667999170,2cyclictest0-21swapper/210:11:042
2667999170,2cyclictest0-21swapper/210:07:052
2667999170,2cyclictest0-21swapper/208:22:042
2667999170,15cyclictest0-21swapper/210:23:042
26678991714,2cyclictest6268-21kworker/u8:110:43:051
26678991714,2cyclictest1554-21kworker/u8:211:54:041
2667899170,15cyclictest0-21swapper/108:20:121
2667799174,3cyclictest7127-21aten2.4_r3power10:15:150
2667799174,3cyclictest5129-21runrttasks10:00:450
2667799174,3cyclictest32700-21sh11:16:210
2667799174,3cyclictest32051-21tail08:40:170
2667799174,3cyclictest255462chrt11:44:310
2667799174,3cyclictest17542-21cut10:25:220
2667799174,3cyclictest13679-21if_eth007:55:190
2667799174,3cyclictest13679-21if_eth007:55:180
2667799174,3cyclictest10777-21diskmemload12:35:100
2667799174,3cyclictest10777-21diskmemload11:59:510
2667799173,4cyclictest2998-21grep07:30:130
2667799173,3cyclictest23920-21kworker/u8:211:10:160
2667799171,2cyclictest0-21swapper/012:15:070
2667799170,2cyclictest0-21swapper/011:03:190
2667799170,16cyclictest0-21swapper/011:21:210
2667799170,16cyclictest0-21swapper/009:11:310
2667799170,10cyclictest0-21swapper/012:14:140
2668099161,3cyclictest0-21swapper/311:10:123
2668099161,3cyclictest0-21swapper/310:36:363
26680991613,2cyclictest7062-21kworker/u8:010:52:053
26680991613,2cyclictest32065-21kworker/u8:210:56:043
26680991613,2cyclictest31223-21kworker/u8:111:18:053
26680991613,2cyclictest28469-21kworker/u8:307:40:043
26680991613,2cyclictest18256-21kworker/u8:109:20:043
26680991613,2cyclictest18072-21kworker/u8:209:53:053
26680991613,2cyclictest1742-21kworker/u8:209:37:053
2668099161,13cyclictest19021-21ssh11:01:513
2668099160,14cyclictest0-21swapper/312:10:053
2667999160,2cyclictest0-21swapper/211:34:052
2667999160,14cyclictest0-21swapper/208:34:042
26678991613,2cyclictest6836-21kworker/u8:112:08:051
26678991613,2cyclictest4748-21kworker/u8:011:40:051
26678991613,2cyclictest24204-21kworker/u8:010:34:051
26678991613,2cyclictest1554-21kworker/u8:211:55:051
26678991613,2cyclictest12632-21kworker/u8:012:10:051
2667899160,14cyclictest0-21swapper/109:20:121
2667799169,6cyclictest0-21swapper/010:52:290
2667799169,6cyclictest0-21swapper/010:20:490
2667799165,2cyclictest10777-21diskmemload09:55:040
2667799164,3cyclictest9086-21aten2.4_r3power09:05:130
2667799164,3cyclictest5129-21runrttasks07:11:300
2667799163,3cyclictest86162sleep009:03:080
2667799163,3cyclictest7348-21grep07:40:140
2667799163,3cyclictest496-21ssh12:25:490
2667799163,3cyclictest32622-21sh10:41:360
2667799163,3cyclictest29807-21cut09:30:130
2667799163,3cyclictest26676-21cyclictest08:45:130
2667799161,4cyclictest30724-21sendmail07:20:010
2667799161,4cyclictest30681-21rm10:05:180
2667799161,4cyclictest2306-21on_ac_power07:30:010
2667799161,4cyclictest18022-21proc_pri08:05:200
2667799161,3cyclictest0-21swapper/010:55:170
2667799161,3cyclictest0-21swapper/008:10:120
2667799160,15cyclictest0-21swapper/007:50:170
2668099151,3cyclictest516-21cron08:45:003
26680991512,2cyclictest6293-21kworker/u8:108:37:053
26680991512,2cyclictest6293-21kworker/u8:107:51:043
26680991512,2cyclictest32674-21kworker/u8:008:58:043
26680991512,2cyclictest29005-21kworker/u8:011:57:043
26680991512,2cyclictest28469-21kworker/u8:307:33:053
26680991512,2cyclictest1742-21kworker/u8:209:44:043
26680991512,2cyclictest15928-21kworker/u8:111:52:053
26680991512,2cyclictest0-21swapper/309:57:283
2668099151,12cyclictest25027-21apt-get09:25:133
2668099150,2cyclictest1043-21ssh10:43:053
2668099150,13cyclictest0-21swapper/311:40:053
26679991512,2cyclictest26676-21cyclictest12:13:302
2667999150,2cyclictest0-21swapper/207:50:042
26678991512,2cyclictest9654-21kworker/u8:211:29:041
26678991512,2cyclictest6293-21kworker/u8:107:53:051
26678991512,2cyclictest4748-21kworker/u8:011:23:051
26678991512,2cyclictest31223-21kworker/u8:111:17:051
26678991512,2cyclictest26750-21kworker/u8:007:22:051
26678991512,2cyclictest10777-21diskmemload10:48:321
2667899150,13cyclictest16965-21missed_timers09:50:181
2667899150,13cyclictest0-21swapper/112:25:321
2667899150,13cyclictest0-21swapper/111:06:031
2667899150,13cyclictest0-21swapper/110:56:531
2667899150,13cyclictest0-21swapper/110:20:011
2667899150,13cyclictest0-21swapper/109:26:031
2667899150,13cyclictest0-21swapper/107:40:121
2667799154,3cyclictest3038-21munin-node09:35:060
2667799154,2cyclictest0-21swapper/010:30:160
2667799153,3cyclictest4795-21cpu07:35:140
2667799153,3cyclictest2817-21cut08:50:140
2667799153,3cyclictest24808-21taskset08:23:380
2667799153,3cyclictest15550-21fschecks_time08:00:140
2667799153,3cyclictest15550-21fschecks_time08:00:130
2667799152,3cyclictest32762-21sed07:25:000
2667799152,11cyclictest0-21swapper/007:50:000
26680991411,2cyclictest6836-21kworker/u8:112:15:053
26680991411,2cyclictest6293-21kworker/u8:107:45:053
26680991411,2cyclictest4748-21kworker/u8:011:24:013
26680991411,2cyclictest4311-21kworker/u8:109:05:043
26680991411,2cyclictest32674-21kworker/u8:009:02:053
26680991411,2cyclictest24677-21kworker/u8:208:30:043
26680991411,2cyclictest16736-21kworker/u8:208:10:023
26680991411,2cyclictest11968-21kworker/u8:111:07:053
26680991411,2cyclictest0-21swapper/311:35:283
2668099140,13cyclictest0-21swapper/310:20:153
2668099140,12cyclictest10239-21ssh12:01:453
2668099140,12cyclictest0-21swapper/310:00:043
2667999140,13cyclictest0-21swapper/211:22:302
26678991411,2cyclictest9210-21ssh10:51:021
26678991411,2cyclictest6293-21kworker/u8:107:46:051
26678991411,2cyclictest32674-21kworker/u8:008:59:041
26678991411,2cyclictest26750-21kworker/u8:007:28:021
26678991411,2cyclictest26750-21kworker/u8:007:18:041
26678991411,2cyclictest19899-21kworker/u8:212:39:041
26678991411,2cyclictest0-21swapper/111:31:261
26678991411,2cyclictest0-21swapper/109:55:021
26678991411,2cyclictest0-21swapper/109:35:121
26678991411,2cyclictest0-21swapper/107:55:291
26678991411,2cyclictest0-21swapper/107:55:281
2667899140,2cyclictest0-21swapper/110:11:571
2667899140,2cyclictest0-21swapper/109:10:041
2667899140,13cyclictest0-21swapper/110:24:421
2667899140,13cyclictest0-21swapper/109:46:221
2667899140,12cyclictest21489-21apt-get12:15:061
2667899140,12cyclictest0-21swapper/112:02:371
2667899140,12cyclictest0-21swapper/111:35:121
2667899140,12cyclictest0-21swapper/110:25:151
2667899140,12cyclictest0-21swapper/109:19:561
2667899140,12cyclictest0-21swapper/107:10:181
2667799144,2cyclictest6655-21kworker/0:108:15:020
2667799144,2cyclictest0-21swapper/008:26:470
26680991310,2cyclictest32674-21kworker/u8:008:46:053
26680991310,2cyclictest20276-21kworker/u8:107:10:023
2668099130,12cyclictest0-21swapper/312:35:163
2668099130,12cyclictest0-21swapper/307:22:053
2667999130,2cyclictest0-21swapper/208:50:052
2667999130,12cyclictest0-21swapper/211:17:012
2667999130,11cyclictest32065-21kworker/u8:210:42:052
26678991310,2cyclictest6293-21kworker/u8:108:36:011
26678991310,2cyclictest0-21swapper/110:05:251
26678991310,2cyclictest0-21swapper/110:00:021
2667899130,2cyclictest0-21swapper/110:35:221
2667899130,2cyclictest0-21swapper/109:33:041
2667899130,2cyclictest0-21swapper/109:05:141
2667899130,2cyclictest0-21swapper/108:45:131
2667899130,12cyclictest10777-21diskmemload11:11:261
2667899130,12cyclictest0-21swapper/111:01:021
2667899130,12cyclictest0-21swapper/109:40:481
2667799131,3cyclictest5237-21diskstats08:55:150
2668099129,2cyclictest6293-21kworker/u8:108:20:043
2668099129,2cyclictest4311-21kworker/u8:108:53:053
2668099120,2cyclictest0-21swapper/307:28:013
2668099120,2cyclictest0-21swapper/307:18:043
2667999129,2cyclictest0-21swapper/212:34:272
2667999120,2cyclictest0-21swapper/210:35:152
2667999120,10cyclictest0-21swapper/207:45:022
2667899129,2cyclictest6293-21kworker/u8:108:33:041
2667899129,2cyclictest6293-21kworker/u8:108:15:051
2667899129,2cyclictest6293-21kworker/u8:107:37:051
2667899129,2cyclictest0-21swapper/108:10:221
2667899129,2cyclictest0-21swapper/108:00:271
2667899129,2cyclictest0-21swapper/108:00:261
2667899120,11cyclictest0-21swapper/108:40:141
211322110,2sleep30-21swapper/308:15:123
2668099107,2cyclictest0-21swapper/307:55:173
2668099107,2cyclictest0-21swapper/307:55:173
2667999100,8cyclictest0-21swapper/208:09:552
2667899100,2cyclictest0-21swapper/108:07:041
266809996,2cyclictest0-21swapper/308:25:103
266809990,2cyclictest0-21swapper/308:00:313
266809990,2cyclictest0-21swapper/308:00:303
266809990,2cyclictest0-21swapper/307:37:053
266789996,2cyclictest2763-21apt-get08:50:141
266789990,7cyclictest6979-21munin-node1
266789990,2cyclictest2519-21munin-node1
266789990,2cyclictest0-21swapper/108:30:011
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional