You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-05-13 - 02:56

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #3, slot #6

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack3slot6s.osadl.org (updated Tue May 13, 2025 00:43:49)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
27582116105,6sleep30-21swapper/319:05:543
2924210897,6sleep00-21swapper/019:07:510
1268210897,6sleep10-21swapper/119:05:041
2990210695,6sleep20-21swapper/219:08:412
82002820,1sleep0101ktimersoftd/019:20:200
94132800,0sleep30-21swapper/319:24:593
306542800,0sleep00-21swapper/021:53:350
306542800,0sleep00-21swapper/021:53:340
283932720,1sleep0101ktimersoftd/021:18:390
317862700,0sleep30-21swapper/322:26:453
239632690,0sleep20-21swapper/222:19:552
213122230,1sleep0101ktimersoftd/023:55:000
313399211,2cyclictest0-21swapper/120:20:131
313399210,19cyclictest0-21swapper/120:15:091
177542210,0sleep10-21swapper/122:12:591
313499203,2cyclictest0-21swapper/223:03:032
3134992017,2cyclictest7458-21kworker/u8:021:10:052
313399204,4cyclictest0-21swapper/123:39:061
256422200,0sleep10-21swapper/121:48:271
229792200,1sleep2301ktimersoftd/221:45:192
313599191,2cyclictest28668-21ssh21:19:053
3134991916,2cyclictest5109-21kworker/u8:220:34:052
3134991916,2cyclictest32137-21kworker/u8:022:10:042
3134991916,2cyclictest1398-21kworker/u8:000:37:052
3134991916,2cyclictest1398-21kworker/u8:000:25:032
3132991916,2cyclictest16402-21kworker/u8:200:23:050
124952190,1sleep00-21swapper/023:45:190
3134991815,2cyclictest5193-21kworker/u8:019:22:052
3134991815,2cyclictest4819-21kworker/u8:123:42:042
3134991815,2cyclictest29881-21kworker/u8:022:58:042
3134991815,2cyclictest20102-21kworker/u8:121:58:042
3134991815,2cyclictest1398-21kworker/u8:000:14:052
3134991815,2cyclictest12973-21kworker/u8:019:36:032
3134991815,2cyclictest12201-21kworker/u8:222:24:042
313499181,4cyclictest0-21swapper/223:56:052
3134991814,3cyclictest31824-21kworker/u8:020:28:032
313499181,2cyclictest0-21swapper/223:35:042
313399185,11cyclictest18743-21latency_hist00:25:011
313399184,12cyclictest29893-21cut00:35:251
3133991813,3cyclictest0-21swapper/119:45:201
3132991815,2cyclictest32137-21kworker/u8:022:08:050
237602180,1sleep00-21swapper/022:19:360
122292180,0sleep1221ktimersoftd/123:12:251
3135991714,2cyclictest20102-21kworker/u8:121:44:053
3134991714,2cyclictest9856-21kworker/u8:223:45:052
3134991714,2cyclictest7458-21kworker/u8:021:09:052
3134991714,2cyclictest7458-21kworker/u8:020:48:042
3134991714,2cyclictest5193-21kworker/u8:019:17:052
3134991714,2cyclictest29881-21kworker/u8:023:10:042
3134991714,2cyclictest23784-21kworker/u8:222:51:042
3134991714,2cyclictest23697-21kworker/u8:023:30:052
3134991714,2cyclictest19749-21kworker/u8:120:12:042
3134991714,2cyclictest15639-21kworker/u8:220:20:052
3134991714,2cyclictest15364-21kworker/u8:123:53:032
3134991714,2cyclictest12973-21kworker/u8:019:47:052
3134991714,2cyclictest11651-21kworker/u8:221:00:042
3134991714,2cyclictest11651-21kworker/u8:220:54:032
313399175,10cyclictest19377-21turbostat.cron21:10:001
313399175,10cyclictest0-21swapper/121:50:231
313399175,10cyclictest0-21swapper/121:50:221
313399172,13cyclictest0-21swapper/120:55:201
3133991714,2cyclictest0-21swapper/123:41:261
313399170,15cyclictest0-21swapper/121:42:161
3132991714,2cyclictest2579-21kworker/u8:223:03:030
3132991714,2cyclictest25383-21kworker/u8:221:48:040
3135991613,2cyclictest1398-21kworker/u8:000:26:043
313599161,2cyclictest12221-21apt-get21:35:133
3135991611,3cyclictest3649-21taskset23:04:333
313599160,14cyclictest8146-21ssh22:03:243
313599160,14cyclictest0-21swapper/322:10:133
313599160,14cyclictest0-21swapper/319:30:133
3134991613,2cyclictest9532-21kworker/u8:123:22:042
3134991613,2cyclictest9154-21kworker/u8:219:25:052
3134991613,2cyclictest7291-21kworker/u8:021:35:052
3134991613,2cyclictest29881-21kworker/u8:023:07:042
3134991613,2cyclictest2770-21kworker/u8:120:36:042
3134991613,2cyclictest21976-21kworker/u8:100:31:042
3134991613,2cyclictest20102-21kworker/u8:121:52:042
3134991613,2cyclictest20102-21kworker/u8:121:52:032
3134991613,2cyclictest19749-21kworker/u8:120:02:042
3134991613,2cyclictest11651-21kworker/u8:221:22:052
3134991613,2cyclictest11651-21kworker/u8:220:58:042
313499161,13cyclictest7250-21apt-get21:30:132
313499160,2cyclictest0-21swapper/222:49:042
313399169,5cyclictest0-21swapper/123:00:221
313399165,9cyclictest11099-21unixbench_singl22:05:281
313399164,9cyclictest7042-21munin-node1
313399164,10cyclictest0-21swapper/123:25:251
313399164,10cyclictest0-21swapper/122:50:181
313399164,10cyclictest0-21swapper/100:10:151
313399163,11cyclictest0-21swapper/121:55:251
313399162,12cyclictest0-21swapper/121:13:531
313399162,12cyclictest0-21swapper/119:55:201
3133991613,2cyclictest0-21swapper/121:24:101
3133991613,2cyclictest0-21swapper/120:05:211
3133991613,2cyclictest0-21swapper/119:50:141
313399161,13cyclictest0-21swapper/122:45:081
313599152,9cyclictest4566-21metacity21:56:583
313599151,12cyclictest8990-21apt-get23:10:053
3135991511,2cyclictest22694-21ssh21:12:343
313599150,3cyclictest0-21swapper/323:25:183
313599150,13cyclictest25063-21ssh21:47:343
3134991512,2cyclictest20102-21kworker/u8:122:04:042
313499150,13cyclictest0-21swapper/222:09:042
313499150,13cyclictest0-21swapper/221:25:452
313399159,4cyclictest5190-21turbostat20:35:001
313399156,3cyclictest12327-21df_abs22:40:151
313399154,9cyclictest251512sleep122:20:181
313399154,9cyclictest14317-21munin-node1
313399154,9cyclictest0-21swapper/119:30:261
313399154,4cyclictest0-21swapper/122:55:161
313399154,4cyclictest0-21swapper/120:25:261
313399154,10cyclictest0-21swapper/100:07:051
313399153,10cyclictest0-21swapper/121:30:171
313399153,10cyclictest0-21swapper/120:40:001
313399153,10cyclictest0-21swapper/119:25:141
313399153,10cyclictest0-21swapper/119:15:201
313399152,11cyclictest0-21swapper/123:50:261
313399151,3cyclictest11026-21ssh00:16:081
313399150,13cyclictest0-21swapper/122:15:181
3132991512,2cyclictest29881-21kworker/u8:022:59:050
3132991512,2cyclictest2579-21kworker/u8:223:35:050
3132991512,2cyclictest21482-21kworker/u8:023:56:050
3135991411,2cyclictest27802-21kworker/u8:119:18:043
3135991411,2cyclictest23784-21kworker/u8:222:57:053
3135991411,2cyclictest0-21swapper/323:34:143
3135991411,2cyclictest0-21swapper/322:50:243
3135991410,2cyclictest11264-21ssh00:16:293
313599140,3cyclictest27574-21perl21:50:143
313599140,3cyclictest27574-21perl21:50:133
313599140,2cyclictest0-21swapper/322:45:133
313599140,13cyclictest0-21swapper/323:56:143
313599140,13cyclictest0-21swapper/322:40:043
313599140,12cyclictest10236-21ssh23:43:253
313599140,12cyclictest0-21swapper/321:25:163
313599140,12cyclictest0-21swapper/300:10:243
313499140,2cyclictest0-21swapper/221:40:022
313499140,13cyclictest0-21swapper/200:17:032
313499140,12cyclictest27504-21ssh21:17:192
313399144,8cyclictest0-21swapper/119:35:171
313399144,4cyclictest0-21swapper/122:00:171
313399143,9cyclictest4503-21aten2.4_r3power23:05:161
313399143,9cyclictest3037-21pzem_power22:30:231
313399143,9cyclictest19379-21diskmemload00:30:221
313399143,9cyclictest0-21swapper/123:45:141
313399143,9cyclictest0-21swapper/119:20:171
313399143,4cyclictest0-21swapper/121:35:181
313399142,2cyclictest0-21swapper/121:29:471
313399142,2cyclictest0-21swapper/100:27:381
313399142,2cyclictest0-21swapper/100:04:361
313399142,10cyclictest0-21swapper/122:25:281
3133991410,3cyclictest0-21swapper/121:15:081
313399140,12cyclictest0-21swapper/123:34:131
3132991411,2cyclictest32479-21kworker/u8:100:39:040
3132991411,2cyclictest0-21swapper/020:55:180
313299140,2cyclictest0-21swapper/021:35:140
313299140,2cyclictest0-21swapper/021:29:280
313299140,2cyclictest0-21swapper/000:06:230
313299140,13cyclictest0-21swapper/020:25:270
313299140,12cyclictest0-21swapper/000:10:270
284582140,2sleep00-21swapper/023:29:350
211142140,1sleep30-21swapper/323:20:553
3135991310,2cyclictest7458-21kworker/u8:021:04:033
3135991310,2cyclictest12201-21kworker/u8:222:38:043
3135991310,2cyclictest12201-21kworker/u8:222:17:043
3135991310,2cyclictest11651-21kworker/u8:221:31:053
3135991310,2cyclictest0-21swapper/322:06:243
3135991310,2cyclictest0-21swapper/320:35:213
3135991310,2cyclictest0-21swapper/319:10:123
3135991310,2cyclictest0-21swapper/300:38:373
3135991310,2cyclictest0-21swapper/300:21:443
313599130,2cyclictest4551-21ssh23:37:203
313599130,2cyclictest0-21swapper/322:22:063
313599130,2cyclictest0-21swapper/300:30:313
313599130,2cyclictest0-21swapper/300:06:243
313599130,12cyclictest0-21swapper/323:50:163
313599130,12cyclictest0-21swapper/323:47:453
313599130,12cyclictest0-21swapper/323:06:043
313599130,12cyclictest0-21swapper/322:31:473
313599130,11cyclictest0-21swapper/300:00:493
313499131,3cyclictest9721-21ssh22:37:072
3134991310,2cyclictest0-21swapper/223:25:502
3134991310,2cyclictest0-21swapper/222:44:012
313499130,2cyclictest0-21swapper/222:25:262
313499130,2cyclictest0-21swapper/219:41:052
313499130,2cyclictest0-21swapper/200:21:032
313499130,1cyclictest0-21swapper/200:07:232
313499130,12cyclictest0-21swapper/222:31:152
313499130,12cyclictest0-21swapper/219:30:152
313499130,12cyclictest0-21swapper/200:00:172
313399133,8cyclictest0-21swapper/123:20:221
313399130,2cyclictest0-21swapper/119:40:181
313399130,11cyclictest0-21swapper/120:10:251
313299131,11cyclictest0-21swapper/000:15:060
3132991310,2cyclictest7458-21kworker/u8:021:05:040
3132991310,2cyclictest19379-21diskmemload22:47:480
3132991310,2cyclictest19379-21diskmemload00:04:580
3132991310,2cyclictest0-21swapper/023:32:280
3132991310,2cyclictest0-21swapper/020:40:130
313299130,2cyclictest0-21swapper/022:53:570
313299130,2cyclictest0-21swapper/000:27:390
313299130,12cyclictest25347-21ssh00:30:280
313299130,12cyclictest0-21swapper/021:14:280
313299130,11cyclictest0-21swapper/022:00:380
80762120,1sleep00-21swapper/021:30:180
313599129,2cyclictest31824-21kworker/u8:020:33:043
313599129,2cyclictest31824-21kworker/u8:020:22:043
313599129,2cyclictest27802-21kworker/u8:119:35:043
313599129,2cyclictest15639-21kworker/u8:219:59:053
313599129,2cyclictest15639-21kworker/u8:219:42:043
313599129,2cyclictest0-21swapper/320:45:223
313599129,2cyclictest0-21swapper/320:40:273
313599128,3cyclictest19749-21kworker/u8:119:54:053
313599122,7cyclictest0-21swapper/323:16:593
313599120,2cyclictest0-21swapper/321:20:273
313599120,11cyclictest0-21swapper/319:45:253
313599120,11cyclictest0-21swapper/319:25:203
313499120,2cyclictest0-21swapper/223:15:042
313499120,1cyclictest0-21swapper/219:55:122
313499120,11cyclictest0-21swapper/220:08:052
313399125,5cyclictest8319-21needreboot20:40:191
313399125,5cyclictest11805-21tr20:50:151
313399123,4cyclictest0-21swapper/123:55:011
313399120,11cyclictest0-21swapper/120:00:181
313399120,10cyclictest0-21swapper/121:00:201
313299129,2cyclictest0-21swapper/023:44:380
313299129,2cyclictest0-21swapper/023:16:180
313299129,2cyclictest0-21swapper/022:42:500
313299129,2cyclictest0-21swapper/022:36:390
313299129,2cyclictest0-21swapper/022:30:300
313299129,2cyclictest0-21swapper/022:12:080
313299129,2cyclictest0-21swapper/021:43:170
313299120,2cyclictest0-21swapper/023:10:270
313299120,2cyclictest0-21swapper/019:50:180
313299120,11cyclictest0-21swapper/023:20:090
313299120,11cyclictest0-21swapper/023:05:080
313299120,11cyclictest0-21swapper/022:29:580
313299120,11cyclictest0-21swapper/022:20:090
313299120,11cyclictest0-21swapper/021:55:080
313299120,11cyclictest0-21swapper/020:15:090
313299120,10cyclictest30909-21ssh21:20:270
313599118,2cyclictest15639-21kworker/u8:220:05:043
313599118,2cyclictest15639-21kworker/u8:220:04:043
313599118,2cyclictest11651-21kworker/u8:220:55:053
313599118,2cyclictest11651-21kworker/u8:220:53:053
313599118,2cyclictest0-21swapper/321:05:193
313599112,6cyclictest0-21swapper/320:13:013
313499110,2cyclictest0-21swapper/219:10:042
313399115,4cyclictest9547-21aten2.4-expect20:45:161
313399113,4cyclictest0-21swapper/119:10:131
313299118,2cyclictest9154-21kworker/u8:219:28:040
313299118,2cyclictest0-21swapper/019:33:400
313299110,9cyclictest0-21swapper/020:05:150
313599101,6cyclictest0-21swapper/320:25:273
313299107,2cyclictest2770-21kworker/u8:120:37:050
313299107,2cyclictest15639-21kworker/u8:219:41:050
313299100,8cyclictest4359-21pzem_current20:30:210
31359991,5cyclictest0-21swapper/320:15:103
31349996,2cyclictest0-21swapper/220:40:282
31349994,3cyclictest30287-21interrupts20:15:142
31329996,2cyclictest19749-21kworker/u8:119:57:050
31329996,2cyclictest15675-21aten2.4-expect21:00:130
31329996,2cyclictest0-21swapper/020:10:290
31329995,3cyclictest101ktimersoftd/020:20:130
31329994,3cyclictest12205-21kernelversion20:50:180
31329994,3cyclictest10613-21awk20:45:280
31329990,8cyclictest0-21swapper/019:45:250
31349983,3cyclictest20397-21wc19:50:202
31329985,2cyclictest2936-21kworker/u8:219:10:050
31329984,3cyclictest24487-21ntpdc20:00:180
31329974,2cyclictest19750irq/25-ahci[00019:15:000
31329970,6cyclictest0-21swapper/019:35:080
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional