You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-12-01 - 00:24

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #3, slot #6

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rack3slot6s.osadl.org (updated Sun Nov 30, 2025 12:43:49)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
162121390,0sleep30-21swapper/311:20:163
24747211894,18sleep30-21swapper/307:05:293
24942211698,12sleep20-21swapper/207:08:012
24992210687,12sleep10-21swapper/107:08:401
24868210594,6sleep00-21swapper/007:07:020
221272690,0sleep30-21swapper/311:07:053
2513299230,18cyclictest0-21swapper/309:56:043
2513299230,18cyclictest0-21swapper/307:14:043
32632220,1sleep10-21swapper/110:12:151
2513299220,20cyclictest28642-21ssh10:40:113
2513299220,18cyclictest0-21swapper/307:22:013
2513299220,17cyclictest0-21swapper/311:31:043
25131992219,2cyclictest5436-21kworker/u8:109:04:012
25131992219,2cyclictest29051-21kworker/u8:208:37:042
25131992219,2cyclictest2528-21kworker/u8:107:33:042
25131992219,2cyclictest22930-21kworker/u8:110:10:052
2513099223,2cyclictest0-21swapper/109:49:041
2513099223,2cyclictest0-21swapper/109:34:041
2513099223,2cyclictest0-21swapper/107:24:041
2513099223,18cyclictest0-21swapper/112:30:011
2513099222,2cyclictest0-21swapper/111:02:041
2513099220,5cyclictest0-21swapper/112:09:041
151032220,1sleep20-21swapper/209:50:152
123642220,0sleep30-21swapper/309:12:323
2513299210,19cyclictest0-21swapper/309:15:143
2513299210,17cyclictest0-21swapper/308:11:043
2513299210,17cyclictest0-21swapper/307:42:043
2513299210,16cyclictest0-21swapper/309:35:043
25131992118,2cyclictest8951-21kworker/u8:208:14:012
25131992118,2cyclictest8951-21kworker/u8:207:48:042
25131992118,2cyclictest2684-21kworker/u8:110:53:042
25131992118,2cyclictest22229-21kworker/u8:009:30:042
25131992118,2cyclictest19191-21kworker/u8:008:45:042
25131992118,2cyclictest19191-21kworker/u8:008:34:042
25131992118,2cyclictest11480-21kworker/u8:008:04:032
2513099212,4cyclictest0-21swapper/107:53:041
2513099212,2cyclictest0-21swapper/112:18:031
2513099212,2cyclictest0-21swapper/111:54:051
2513099212,2cyclictest0-21swapper/110:39:041
2513099212,2cyclictest0-21swapper/110:17:041
2513099212,2cyclictest0-21swapper/109:50:051
2513099212,2cyclictest0-21swapper/109:15:031
2513099212,2cyclictest0-21swapper/107:34:041
2513099212,18cyclictest0-21swapper/111:20:011
2513099212,16cyclictest0-21swapper/108:18:011
25130992118,2cyclictest19191-21kworker/u8:008:20:041
2513099211,2cyclictest23-21ksoftirqd/112:00:041
2513099210,4cyclictest0-21swapper/111:56:051
2513099210,4cyclictest0-21swapper/111:56:051
25129992117,2cyclictest0-21swapper/009:25:110
25129992117,2cyclictest0-21swapper/009:25:110
124912210,0sleep20-21swapper/211:30:482
25132992017,2cyclictest17738-21kworker/u8:010:31:043
25132992017,2cyclictest1020-21kworker/u8:109:51:043
2513299200,18cyclictest0-21swapper/312:30:123
2513299200,18cyclictest0-21swapper/310:45:143
2513299200,16cyclictest0-21swapper/307:46:043
25131992017,2cyclictest8951-21kworker/u8:207:50:042
25131992017,2cyclictest29051-21kworker/u8:208:41:042
25131992017,2cyclictest2684-21kworker/u8:110:46:042
25131992017,2cyclictest12434-21kworker/u8:011:35:042
25131992017,2cyclictest1204-21kworker/u8:012:05:042
25131992017,2cyclictest11480-21kworker/u8:007:57:042
2513099202,16cyclictest0-21swapper/110:25:041
25130992017,2cyclictest2684-21kworker/u8:110:57:031
25130992017,2cyclictest25074-21kworker/u8:108:29:041
25130992017,2cyclictest11480-21kworker/u8:007:56:041
2513099201,2cyclictest0-21swapper/111:47:041
2513099201,2cyclictest0-21swapper/110:52:041
2513099201,2cyclictest0-21swapper/109:00:041
2513099201,2cyclictest0-21swapper/108:01:041
2513099201,2cyclictest0-21swapper/107:13:041
25129992015,3cyclictest5315-21ssh10:15:110
25132991916,2cyclictest7753-21kworker/u8:010:56:043
25132991916,2cyclictest13727-21kworker/u8:211:18:043
2513299190,18cyclictest0-21swapper/310:28:053
2513299190,18cyclictest0-21swapper/310:11:043
2513299190,18cyclictest0-21swapper/310:09:043
2513299190,18cyclictest0-21swapper/309:20:043
2513299190,18cyclictest0-21swapper/309:20:033
2513299190,18cyclictest0-21swapper/307:16:013
2513299190,15cyclictest0-21swapper/312:11:043
2513299190,14cyclictest0-21swapper/312:22:043
25131991916,2cyclictest29073-21kworker/u8:107:21:042
25131991916,2cyclictest28326-21kworker/u8:111:56:012
25131991916,2cyclictest28326-21kworker/u8:111:56:002
25131991916,2cyclictest2528-21kworker/u8:107:36:042
25131991916,2cyclictest22930-21kworker/u8:110:08:032
25131991916,2cyclictest19191-21kworker/u8:008:24:042
25131991916,2cyclictest18469-21kworker/u8:111:07:032
2513199190,2cyclictest0-21swapper/209:49:042
2513099191,2cyclictest0-21swapper/111:29:041
2513099191,2cyclictest0-21swapper/110:48:041
2513099191,2cyclictest0-21swapper/110:03:041
2513099190,2cyclictest0-21swapper/112:12:041
2513099190,2cyclictest0-21swapper/109:12:041
2513099190,2cyclictest0-21swapper/108:51:041
2513099190,2cyclictest0-21swapper/108:43:041
2513099190,2cyclictest0-21swapper/108:12:041
2513099190,2cyclictest0-21swapper/108:07:041
2513099190,2cyclictest0-21swapper/107:28:031
2512999194,13cyclictest28600-21sh11:14:560
25129991913,4cyclictest19286-21latency_hist08:15:000
57962180,0sleep30-21swapper/309:40:143
2513299180,17cyclictest0-21swapper/308:58:043
2513299180,17cyclictest0-21swapper/308:50:043
25131991815,2cyclictest23720-21kworker/u8:112:26:042
25131991815,2cyclictest16846-21kworker/u8:109:21:042
25131991815,2cyclictest16846-21kworker/u8:109:21:042
25131991815,2cyclictest15018-21kworker/u8:007:29:042
25131991815,2cyclictest15018-21kworker/u8:007:17:042
25131991815,2cyclictest13523-21kworker/u8:309:58:042
2513199180,2cyclictest0-21swapper/212:18:032
2513199180,2cyclictest0-21swapper/211:54:052
2513199180,2cyclictest0-21swapper/211:02:042
2513199180,2cyclictest0-21swapper/210:39:042
2513199180,2cyclictest0-21swapper/209:15:042
25130991815,2cyclictest3286-21kworker/u8:309:39:041
2513099181,2cyclictest0-21swapper/111:39:041
2513099181,2cyclictest0-21swapper/109:08:041
2513099180,2cyclictest0-21swapper/110:44:051
2512999183,13cyclictest0-21swapper/011:16:200
25129991813,3cyclictest31605-21sh10:42:480
2512999181,2cyclictest0-21swapper/008:20:120
25132991714,2cyclictest23720-21kworker/u8:112:35:043
2513299170,16cyclictest0-21swapper/311:44:043
2513299170,16cyclictest0-21swapper/310:24:043
25131991714,2cyclictest16962-21kworker/u8:311:46:042
2513199170,2cyclictest0-21swapper/212:34:042
2513199170,2cyclictest0-21swapper/212:00:042
2513199170,2cyclictest0-21swapper/211:29:042
2513199170,2cyclictest0-21swapper/211:23:042
2513199170,2cyclictest0-21swapper/210:17:042
2513199170,2cyclictest0-21swapper/210:03:042
2513099170,2cyclictest0-21swapper/111:13:041
2513099170,2cyclictest0-21swapper/108:33:041
2513099170,2cyclictest0-21swapper/107:43:041
2513099170,16cyclictest0-21swapper/109:23:041
2513099170,16cyclictest0-21swapper/109:23:041
2512999173,12cyclictest11623-21perl12:05:150
2512999171,13cyclictest6567-21grep07:40:280
25132991613,2cyclictest9009-21kworker/u8:212:06:013
25132991613,2cyclictest22476-21kworker/u8:210:36:043
2513299160,15cyclictest0-21swapper/308:25:043
25131991613,2cyclictest1262-21kworker/u8:208:52:032
2513199160,2cyclictest0-21swapper/210:29:042
2513099163,2cyclictest0-21swapper/112:39:041
2513099161,3cyclictest0-21swapper/110:24:521
2513099161,2cyclictest0-21swapper/108:46:041
2513099160,2cyclictest0-21swapper/112:28:041
2513099160,2cyclictest0-21swapper/109:25:041
2513099160,2cyclictest0-21swapper/109:25:041
2513099160,2cyclictest0-21swapper/108:55:041
2513099160,15cyclictest0-21swapper/109:55:041
2512999163,2cyclictest0-21swapper/010:04:170
2512999162,12cyclictest9214-21diskmemload11:01:020
25129991612,3cyclictest0-21swapper/007:25:170
2512999161,13cyclictest9214-21diskmemload10:35:160
2512999161,13cyclictest0-21swapper/011:41:480
25132991512,2cyclictest15018-21kworker/u8:007:26:043
2513299150,2cyclictest0-21swapper/308:05:043
25131991511,2cyclictest7171-21apt-get09:05:062
2513199150,2cyclictest0-21swapper/210:55:042
2513099150,2cyclictest0-21swapper/107:47:041
2512999152,7cyclictest0-21swapper/012:25:010
2512999151,4cyclictest0-21swapper/012:12:150
2512999151,3cyclictest0-21swapper/011:55:410
2512999151,3cyclictest0-21swapper/011:55:410
25129991511,3cyclictest0-21swapper/012:00:330
25129991511,3cyclictest0-21swapper/010:51:130
25129991511,3cyclictest0-21swapper/009:57:590
2512999151,12cyclictest0-21swapper/011:53:060
2512999151,12cyclictest0-21swapper/010:26:040
25129991510,4cyclictest0-21swapper/010:09:450
25129991510,3cyclictest24722-21ssh11:45:110
2512999150,13cyclictest10743-21ntpq09:45:180
2512999150,13cyclictest0-21swapper/012:35:070
2512999150,13cyclictest0-21swapper/012:19:180
25132991411,2cyclictest28969-21kworker/u8:012:29:033
25132991411,2cyclictest28326-21kworker/u8:111:56:053
25132991411,2cyclictest28326-21kworker/u8:111:56:053
25132991411,2cyclictest2528-21kworker/u8:107:37:043
25132991411,2cyclictest13727-21kworker/u8:211:02:043
25132991411,2cyclictest13523-21kworker/u8:310:16:033
25132991411,2cyclictest13523-21kworker/u8:309:49:043
25132991410,3cyclictest11480-21kworker/u8:007:59:043
25131991410,2cyclictest26124-21sh11:10:552
2513199140,2cyclictest0-21swapper/212:36:212
2513199140,2cyclictest0-21swapper/209:35:212
2513199140,13cyclictest0-21swapper/211:16:112
2513199140,12cyclictest24694-21ssh09:25:242
2513199140,12cyclictest24694-21ssh09:25:242
2513199140,12cyclictest0-21swapper/212:20:242
2513199140,12cyclictest0-21swapper/210:43:312
25130991411,2cyclictest0-21swapper/109:42:261
2513099140,2cyclictest0-21swapper/111:17:361
2513099140,13cyclictest0-21swapper/112:22:161
2513099140,12cyclictest9214-21diskmemload10:06:441
2513099140,12cyclictest0-21swapper/107:35:191
2512999147,5cyclictest14202-21sh09:15:140
2512999145,7cyclictest4549-21grep08:55:260
2512999142,7cyclictest0-21swapper/008:30:000
2512999142,7cyclictest0-21swapper/007:10:000
2512999142,6cyclictest0-21swapper/007:40:000
2512999142,11cyclictest0-21swapper/010:46:060
2512999141,3cyclictest0-21swapper/012:28:220
2512999141,3cyclictest0-21swapper/009:54:170
25129991411,2cyclictest9214-21diskmemload10:55:560
25129991411,2cyclictest0-21swapper/012:30:230
25129991411,2cyclictest0-21swapper/010:20:170
25129991411,2cyclictest0-21swapper/009:20:490
25129991411,2cyclictest0-21swapper/009:20:490
2512999141,11cyclictest9214-21diskmemload10:32:340
2512999141,11cyclictest0-21swapper/009:35:510
2512999140,13cyclictest0-21swapper/009:41:190
2512999140,12cyclictest0-21swapper/008:40:170
25132991310,2cyclictest2528-21kworker/u8:107:53:043
25132991310,2cyclictest2528-21kworker/u8:107:34:043
25132991310,2cyclictest22343-21kworker/u8:011:45:043
25132991310,2cyclictest22229-21kworker/u8:009:34:043
25132991310,2cyclictest19191-21kworker/u8:008:18:013
25132991310,2cyclictest11480-21kworker/u8:008:02:043
2513299130,12cyclictest0-21swapper/309:04:043
2513299130,11cyclictest0-21swapper/311:50:143
25131991310,2cyclictest0-21swapper/210:33:332
25131991310,2cyclictest0-21swapper/209:10:042
25131991310,2cyclictest0-21swapper/207:10:182
2513199130,2cyclictest0-21swapper/211:42:342
2513199130,11cyclictest0-21swapper/210:22:162
2513099138,3cyclictest20635-21awk10:30:261
25130991310,2cyclictest0-21swapper/111:41:051
2513099130,2cyclictest0-21swapper/111:33:561
2513099130,12cyclictest9214-21diskmemload11:07:091
2512999132,6cyclictest0-21swapper/007:55:000
2512999131,11cyclictest0-21swapper/008:18:380
25129991310,2cyclictest0-21swapper/011:39:150
25129991310,2cyclictest0-21swapper/010:15:010
2512999130,3cyclictest0-21swapper/009:31:210
2512999130,12cyclictest0-21swapper/011:05:460
2512999130,11cyclictest9214-21diskmemload11:20:170
2512999130,11cyclictest0-21swapper/009:10:570
2513299129,2cyclictest6070-21kworker/u8:211:26:043
2513299129,2cyclictest28326-21kworker/u8:112:00:043
2513299129,2cyclictest18859-21kworker/u8:212:18:043
2513299129,2cyclictest0-21swapper/310:02:233
2513299120,11cyclictest0-21swapper/310:50:423
2513199129,2cyclictest19191-21kworker/u8:008:18:052
2513199128,2cyclictest0-21swapper/209:40:152
2513199122,7cyclictest0-21swapper/208:08:592
2513199122,7cyclictest0-21swapper/207:42:592
2513199120,2cyclictest0-21swapper/212:11:052
2513099121,9cyclictest0-21swapper/107:15:171
2513099120,11cyclictest0-21swapper/108:38:391
2512999129,2cyclictest0-21swapper/011:25:190
2512999126,4cyclictest6146-21cat09:00:170
2512999126,4cyclictest4324-21nscd11:31:420
2512999126,4cyclictest26132-21grep08:30:150
2512999126,4cyclictest14356-21ntp_kernel_pll_08:00:180
2512999120,11cyclictest0-21swapper/009:05:260
2512999120,11cyclictest0-21swapper/007:15:140
2513299118,2cyclictest19191-21kworker/u8:008:46:043
2513299118,2cyclictest19191-21kworker/u8:008:43:043
2513299118,2cyclictest1262-21kworker/u8:209:06:043
2513299111,7cyclictest0-21swapper/311:35:593
2513299110,10cyclictest0-21swapper/311:10:023
2513199118,2cyclictest0-21swapper/208:59:392
2513199118,2cyclictest0-21swapper/208:25:172
2512999115,4cyclictest31980-21sed08:45:140
2512999115,4cyclictest29436-21basename07:20:170
2512999115,4cyclictest27585-21aten2.4_r3power08:35:120
2512999115,4cyclictest15649-21egrep08:05:140
2512999115,4cyclictest1116-21munin-node0
2512999114,5cyclictest9733-21cut07:50:140
2512999114,5cyclictest7740-21sed07:45:130
2512999114,5cyclictest1249-21cut08:50:120
2513299100,9cyclictest0-21swapper/309:25:173
2513299100,9cyclictest0-21swapper/309:25:173
251329996,2cyclictest27908-21df_abs08:35:143
251329996,2cyclictest19191-21kworker/u8:008:33:043
251329994,3cyclictest21730-21cpu08:20:133
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional