You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-07-06 - 11:38

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #3, slot #6

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rack3slot6s.osadl.org (updated Sun Jul 06, 2025 00:43:50)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
296142115104,6sleep30-21swapper/319:06:013
29684211193,12sleep10-21swapper/119:06:561
86821080,1sleep3866-21irqstats22:35:183
29643210883,6sleep00-21swapper/019:06:230
29660210789,12sleep20-21swapper/219:06:362
2995499340,17cyclictest0-21swapper/323:10:123
2995199290,13cyclictest0-21swapper/020:00:140
120602240,1sleep337-21rcuc/321:40:183
69032220,1sleep10-21swapper/122:40:341
2995499210,19cyclictest0-21swapper/300:25:123
2995299210,19cyclictest332250irq/24-eth100:15:131
2995199210,16cyclictest0-21swapper/022:44:050
2995199210,16cyclictest0-21swapper/022:31:050
2995199210,16cyclictest0-21swapper/021:11:050
29952992015,3cyclictest332250irq/24-eth122:45:141
2995299200,18cyclictest0-21swapper/100:25:111
2995199200,2cyclictest0-21swapper/019:45:110
2995199200,2cyclictest0-21swapper/019:20:110
2995499190,17cyclictest0-21swapper/320:55:133
29953991915,3cyclictest20935-21kworker/u8:022:01:052
2995399191,2cyclictest0-21swapper/221:14:052
2995199190,18cyclictest0-21swapper/023:55:130
2995199190,17cyclictest0-21swapper/000:00:150
2995399182,2cyclictest29013-1kworker/u9:123:51:052
2995399182,2cyclictest0-21swapper/223:14:052
29953991815,2cyclictest8227-21kworker/u8:022:24:052
29953991815,2cyclictest5602-21kworker/u8:119:53:052
29953991815,2cyclictest5602-21kworker/u8:119:37:052
29953991815,2cyclictest5240-21kworker/u8:120:58:052
29953991815,2cyclictest5240-21kworker/u8:120:50:052
29953991815,2cyclictest28111-21kworker/u8:020:30:052
29953991815,2cyclictest28111-21kworker/u8:020:26:022
29953991815,2cyclictest28111-21kworker/u8:020:26:022
29953991815,2cyclictest27678-21kworker/u8:121:57:042
29953991815,2cyclictest26230-21kworker/u8:219:26:052
29953991815,2cyclictest26230-21kworker/u8:219:24:042
29953991815,2cyclictest26230-21kworker/u8:219:16:022
29953991815,2cyclictest20017-21kworker/u8:120:06:022
29953991815,2cyclictest18218-21kworker/u8:121:18:012
2995399181,2cyclictest0-21swapper/222:37:052
2995399181,15cyclictest0-21swapper/223:28:022
2995399181,15cyclictest0-21swapper/223:28:012
2995199180,17cyclictest0-21swapper/021:59:040
29953991714,2cyclictest8011-21kworker/u8:019:45:052
29953991714,2cyclictest31734-21kworker/u8:120:35:052
29953991714,2cyclictest2828-21kworker/u8:023:58:052
29953991714,2cyclictest2828-21kworker/u8:023:46:012
29953991714,2cyclictest27678-21kworker/u8:122:19:052
29953991714,2cyclictest20017-21kworker/u8:120:21:042
29953991714,2cyclictest18218-21kworker/u8:121:20:052
29953991714,2cyclictest17965-21kworker/u8:222:28:052
29953991714,2cyclictest13456-21kworker/u8:220:19:052
29953991714,2cyclictest13077-21kworker/u8:221:52:052
2995399170,2cyclictest0-21swapper/223:19:052
2995399170,2cyclictest0-21swapper/223:19:042
29951991714,2cyclictest6508-21kworker/u8:223:26:060
29951991714,2cyclictest6508-21kworker/u8:223:26:050
29951991714,2cyclictest5248-21kworker/u8:022:50:020
2995199170,16cyclictest0-21swapper/021:40:060
2995199170,15cyclictest0-21swapper/021:45:110
29954991611,3cyclictest30440-21sort22:00:003
29954991611,3cyclictest20772-21memory22:55:173
2995499161,13cyclictest0-21swapper/321:35:143
29953991613,2cyclictest6619-21kworker/u8:000:30:052
29953991613,2cyclictest5240-21kworker/u8:121:05:042
29953991613,2cyclictest29092-21kworker/u8:100:20:042
29953991613,2cyclictest29092-21kworker/u8:100:14:012
29953991613,2cyclictest28727-21kworker/u8:223:37:042
29953991613,2cyclictest1701-21kworker/u8:020:41:042
29953991613,2cyclictest14552-21kworker/u8:119:12:052
29953991613,2cyclictest14031-21kworker/u8:023:21:052
2995399161,2cyclictest23992-21ssh00:05:052
2995399161,2cyclictest0-21swapper/200:28:022
29953991611,3cyclictest22093-21hostname23:30:152
2995399160,2cyclictest0-21swapper/222:45:052
29952991611,3cyclictest0-21swapper/123:35:151
2995199160,2cyclictest0-21swapper/020:17:050
2995199160,15cyclictest0-21swapper/023:20:060
2995499152,7cyclictest0-21swapper/320:45:003
2995499152,7cyclictest0-21swapper/320:15:013
2995499151,12cyclictest14614-21if_err_eth019:50:163
29954991510,3cyclictest0-21swapper/300:20:183
2995499150,13cyclictest24489-21fschecks_count20:15:153
2995499150,13cyclictest13738-21diskmemload21:20:233
2995499150,13cyclictest13640-21sh22:48:233
2995499150,13cyclictest0-21swapper/323:27:343
2995499150,13cyclictest0-21swapper/323:27:333
2995499150,13cyclictest0-21swapper/322:40:063
2995399152,2cyclictest0-21swapper/221:39:052
2995399151,2cyclictest0-21swapper/200:17:052
29953991512,2cyclictest28727-21kworker/u8:223:42:052
2995399150,13cyclictest0-21swapper/220:02:052
29952991512,2cyclictest17591-21ssh00:30:261
2995299151,12cyclictest21085-21ssh22:22:551
2995299150,1cyclictest13738-21diskmemload22:39:071
2995299150,13cyclictest20619-21http_loadtime20:05:161
2995299150,13cyclictest0-21swapper/122:05:251
2995299150,13cyclictest0-21swapper/120:10:191
29951991512,2cyclictest23083-1kworker/u9:223:31:070
29951991512,2cyclictest18218-21kworker/u8:121:50:050
29951991512,2cyclictest18218-21kworker/u8:121:38:050
29951991512,2cyclictest0-21swapper/022:15:180
2995199151,12cyclictest16395-21chrt00:30:110
2995199150,13cyclictest0-21swapper/023:11:370
29954991411,2cyclictest0-21swapper/322:31:523
29954991411,2cyclictest0-21swapper/322:22:433
29954991411,2cyclictest0-21swapper/300:02:473
2995499141,11cyclictest22094-21ssh21:17:543
2995499140,2cyclictest19255-21ssh21:48:033
2995499140,2cyclictest0-21swapper/319:25:233
2995499140,13cyclictest0-21swapper/323:20:233
2995499140,12cyclictest19732-21ssh00:33:063
2995499140,12cyclictest1735-21ssh22:03:253
2995499140,12cyclictest0-21swapper/323:44:073
2995499140,12cyclictest0-21swapper/323:01:333
2995499140,12cyclictest0-21swapper/322:50:043
2995499140,12cyclictest0-21swapper/300:38:523
29953991411,2cyclictest20017-21kworker/u8:120:10:062
2995399140,2cyclictest0-21swapper/220:47:052
2995399140,13cyclictest0-21swapper/222:05:142
2995399140,12cyclictest0-21swapper/221:45:552
2995399140,12cyclictest0-21swapper/200:36:042
2995299149,3cyclictest28730-21rm21:25:131
2995299149,3cyclictest0-21swapper/122:25:151
2995299142,7cyclictest13738-21diskmemload00:10:011
2995299140,2cyclictest0-21swapper/123:15:261
2995299140,2cyclictest0-21swapper/123:15:251
2995299140,2cyclictest0-21swapper/122:15:161
2995299140,13cyclictest0-21swapper/123:25:561
2995299140,13cyclictest0-21swapper/123:25:561
2995299140,13cyclictest0-21swapper/121:19:561
2995299140,12cyclictest5862-21cut19:30:001
2995299140,12cyclictest23218-21ssh00:04:121
2995299140,12cyclictest21099-21sh22:55:231
2995299140,12cyclictest1714-21apt-get20:40:131
2995299140,12cyclictest13738-21diskmemload21:12:261
2995299140,12cyclictest10037-21munin-node1
2995299140,12cyclictest0-21swapper/123:48:101
29951991411,2cyclictest0-21swapper/023:35:260
29951991411,2cyclictest0-21swapper/021:27:480
29951991411,2cyclictest0-21swapper/000:24:270
2995199140,2cyclictest0-21swapper/022:23:170
2995199140,2cyclictest0-21swapper/022:00:580
2995199140,2cyclictest0-21swapper/020:20:180
2995199140,13cyclictest0-21swapper/022:58:080
2995199140,13cyclictest0-21swapper/019:50:180
2995199140,12cyclictest0-21swapper/023:45:140
2995199140,12cyclictest0-21swapper/000:08:100
2995499132,6cyclictest0-21swapper/323:35:013
2995499132,6cyclictest0-21swapper/320:05:013
2995499132,6cyclictest0-21swapper/319:25:003
2995499131,6cyclictest0-21swapper/319:50:013
29954991310,2cyclictest0-21swapper/323:56:053
29954991310,2cyclictest0-21swapper/323:16:573
29954991310,2cyclictest0-21swapper/323:16:563
29954991310,2cyclictest0-21swapper/322:25:143
29954991310,2cyclictest0-21swapper/322:14:153
29954991310,2cyclictest0-21swapper/321:30:143
29954991310,2cyclictest0-21swapper/319:10:123
2995499130,2cyclictest0-21swapper/323:07:043
2995499130,2cyclictest0-21swapper/300:15:563
2995499130,2cyclictest0-21swapper/300:10:183
2995499130,12cyclictest0-21swapper/322:17:223
2995499130,12cyclictest0-21swapper/321:12:593
2995499130,11cyclictest0-21swapper/323:52:243
2995499130,11cyclictest0-21swapper/322:05:543
2995499130,11cyclictest0-21swapper/321:52:533
2995399139,2cyclictest0-21swapper/222:12:092
29953991310,2cyclictest0-21swapper/222:30:162
2995399130,5cyclictest0-21swapper/221:03:052
2995399130,2cyclictest0-21swapper/222:40:052
2995399130,2cyclictest0-21swapper/219:43:052
2995399130,12cyclictest0-21swapper/221:27:252
2995299138,3cyclictest0-21swapper/121:50:261
2995299137,4cyclictest0-21swapper/123:40:311
2995299133,5cyclictest0-21swapper/100:25:011
2995299132,6cyclictest0-21swapper/120:25:001
2995299131,3cyclictest30048-21tr23:05:131
29952991310,2cyclictest0-21swapper/123:58:431
29952991310,2cyclictest0-21swapper/121:49:181
29952991310,2cyclictest0-21swapper/121:44:281
29952991310,2cyclictest0-21swapper/119:35:261
2995299130,2cyclictest0-21swapper/123:30:191
2995299130,2cyclictest0-21swapper/122:50:461
2995299130,2cyclictest0-21swapper/121:21:441
2995299130,2cyclictest0-21swapper/120:30:181
2995299130,12cyclictest0-21swapper/122:11:161
2995299130,12cyclictest0-21swapper/119:15:191
2995299130,11cyclictest0-21swapper/120:50:171
2995199131,2cyclictest26951-21ssh21:22:490
29951991310,2cyclictest7982-21kworker/u8:223:51:040
29951991310,2cyclictest13738-21diskmemload21:19:030
29951991310,2cyclictest13738-21diskmemload00:11:090
29951991310,2cyclictest0-21swapper/023:40:220
29951991310,2cyclictest0-21swapper/023:15:380
29951991310,2cyclictest0-21swapper/023:15:370
29951991310,2cyclictest0-21swapper/023:03:360
29951991310,2cyclictest0-21swapper/022:45:410
29951991310,2cyclictest0-21swapper/022:25:080
29951991310,2cyclictest0-21swapper/019:25:130
29951991310,2cyclictest0-21swapper/000:35:540
2995199130,2cyclictest0-21swapper/023:06:280
2995199130,2cyclictest0-21swapper/022:13:170
2995199130,2cyclictest0-21swapper/000:25:090
2995199130,11cyclictest0-21swapper/022:06:160
297702131,8sleep229839-21ssh23:05:122
22732130,2sleep10-21swapper/121:30:231
2995499129,2cyclictest0-21swapper/323:35:303
2995499129,2cyclictest0-21swapper/320:45:163
2995499129,2cyclictest0-21swapper/320:20:173
2995499129,2cyclictest0-21swapper/319:35:153
2995499122,6cyclictest0-21swapper/319:59:593
2995499120,2cyclictest30599-21ssh21:25:563
2995499120,2cyclictest0-21swapper/300:06:293
2995499120,11cyclictest0-21swapper/323:47:573
2995499120,11cyclictest0-21swapper/320:35:233
2995499120,11cyclictest0-21swapper/320:30:153
2995499120,11cyclictest0-21swapper/320:25:223
2995499120,11cyclictest0-21swapper/320:25:213
2995399120,3cyclictest0-21swapper/219:30:172
2995399120,2cyclictest0-21swapper/223:00:242
2995399120,2cyclictest0-21swapper/221:44:552
2995399120,2cyclictest0-21swapper/200:03:042
2995399120,11cyclictest0-21swapper/219:55:172
2995399120,10cyclictest0-21swapper/222:53:052
2995299129,2cyclictest0-21swapper/123:11:341
2995299129,2cyclictest0-21swapper/119:20:251
2995299127,3cyclictest18026-21munin-node1
2995299121,9cyclictest21342-21munin-node1
2995299121,9cyclictest0-21swapper/119:40:191
2995299121,6cyclictest0-21swapper/119:49:591
2995299121,3cyclictest0-21swapper/119:30:161
2995299120,2cyclictest0-21swapper/121:56:221
2995299120,2cyclictest0-21swapper/120:45:191
2995299120,2cyclictest0-21swapper/119:55:171
2995299120,2cyclictest0-21swapper/119:10:151
2995299120,2cyclictest0-21swapper/100:11:341
2995299120,11cyclictest0-21swapper/123:20:151
2995299120,11cyclictest0-21swapper/122:00:201
2995299120,11cyclictest0-21swapper/121:35:151
2995299120,11cyclictest0-21swapper/120:55:261
2995299120,10cyclictest13738-21diskmemload22:30:441
2995299120,10cyclictest0-21swapper/121:00:191
2995199129,2cyclictest0-21swapper/020:05:130
2995199129,2cyclictest0-21swapper/000:15:420
2995199120,2cyclictest0-21swapper/021:31:020
2995199120,2cyclictest0-21swapper/020:26:220
2995199120,2cyclictest0-21swapper/020:26:220
2995199120,2cyclictest0-21swapper/019:10:140
2995199120,11cyclictest0-21swapper/022:35:100
2995199120,11cyclictest0-21swapper/020:10:150
2995199120,11cyclictest0-21swapper/019:55:200
202942120,1sleep339-21ksoftirqd/320:05:133
2995499113,6cyclictest0-21swapper/319:15:243
2995499112,6cyclictest0-21swapper/319:43:593
2995499110,10cyclictest0-21swapper/321:05:293
2995299118,2cyclictest25844-21ntpq23:00:211
2995299111,8cyclictest32174-21tail20:35:121
2995299110,10cyclictest0-21swapper/121:05:231
2995199118,2cyclictest0-21swapper/021:05:260
2995199118,2cyclictest0-21swapper/019:35:270
2995199118,2cyclictest0-21swapper/019:30:290
2995199110,10cyclictest0-21swapper/019:40:220
2995499104,4cyclictest7240-21sed20:50:253
2995399105,3cyclictest20202-21tr22:55:132
2995399100,2cyclictest0-21swapper/221:30:332
2995299107,2cyclictest0-21swapper/120:15:151
2995199107,2cyclictest31734-21kworker/u8:120:39:050
2995199107,2cyclictest0-21swapper/019:15:160
31050290,1sleep00-21swapper/020:30:190
299549993,3cyclictest10161-21cpuspeed_turbos21:00:123
299549991,7cyclictest0-21swapper/319:33:583
299529995,2cyclictest0-21swapper/119:50:171
299529994,3cyclictest28674-21fschecks_time20:25:201
299529994,3cyclictest28674-21fschecks_time20:25:201
299519996,2cyclictest0-21swapper/020:46:590
299519994,3cyclictest7687-21cron20:55:010
299519990,8cyclictest0-21swapper/021:01:180
299519990,2cyclictest0-21swapper/020:55:060
299519984,2cyclictest2008-21tr20:40:160
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional