You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-19 - 04:28

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #3, slot #6

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rack3slot6s.osadl.org (updated Wed Nov 19, 2025 00:43:50)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
26908211997,17sleep20-21swapper/219:07:342
25796211894,16sleep30-21swapper/319:05:183
27015211692,19sleep00-21swapper/019:08:560
26846211594,16sleep10-21swapper/119:06:441
96522790,1sleep19656-21fschecks_count19:45:171
230642750,1sleep10-21swapper/122:30:161
21902730,0sleep10-21swapper/123:15:291
269122690,0sleep10-21swapper/123:41:501
2712899360,2cyclictest0-21swapper/222:10:142
27127993229,2cyclictest0-21swapper/100:35:151
2712699271,2cyclictest3487-21ssh22:10:140
2712899210,19cyclictest0-21swapper/220:00:152
2712799200,2cyclictest0-21swapper/121:15:121
27129991916,2cyclictest29390-21kworker/u8:223:48:063
2712999190,3cyclictest0-21swapper/321:30:153
27127991916,2cyclictest21558-21kworker/u8:221:22:061
2712799190,18cyclictest0-21swapper/100:15:141
27129991815,2cyclictest7450-21kworker/u8:222:17:053
27129991815,2cyclictest5543-21kworker/u8:023:23:053
27129991815,2cyclictest24676-21kworker/u8:223:02:053
27129991815,2cyclictest15502-21kworker/u8:022:14:053
27129991815,2cyclictest15502-21kworker/u8:022:02:043
27129991815,2cyclictest14522-21kworker/u8:021:37:053
27129991814,2cyclictest5950-21kworker/u8:000:26:013
2712799180,16cyclictest30351-21sh21:30:141
27129991713,2cyclictest29875-21kworker/u8:022:38:013
27129991713,2cyclictest1178-21kworker/u8:122:25:043
2712999170,4cyclictest11340-21cut00:00:133
27129991613,2cyclictest7322-21kworker/u8:221:11:053
27129991613,2cyclictest3290-21kworker/u8:122:59:053
27129991613,2cyclictest20624-21kworker/u8:121:58:053
27129991613,2cyclictest20624-21kworker/u8:121:58:043
27129991613,2cyclictest19187-21kworker/u8:100:14:063
2712899161,13cyclictest25920-21ssh00:15:152
2712899160,13cyclictest29025-21ssh23:44:592
27126991613,2cyclictest13341-21kworker/u8:022:24:060
27126991611,3cyclictest10994-21diskmemload21:40:010
27126991610,5cyclictest0-21swapper/022:26:050
2712699160,14cyclictest0-21swapper/021:22:590
2712699160,11cyclictest0-21swapper/023:18:010
2712999151,3cyclictest20279-21ssh21:19:403
27129991512,2cyclictest26873-21kworker/u8:119:14:013
27129991512,2cyclictest26126-21kworker/u8:222:06:053
27129991512,2cyclictest14522-21kworker/u8:021:20:063
2712999151,12cyclictest5129-21runrttasks00:35:523
2712999151,12cyclictest0-21swapper/322:30:143
27129991510,3cyclictest15919-21ssh00:05:043
2712999150,13cyclictest0-21swapper/323:25:133
2712999150,13cyclictest0-21swapper/322:40:143
2712999150,13cyclictest0-21swapper/321:41:043
2712999150,13cyclictest0-21swapper/321:41:033
2712999150,13cyclictest0-21swapper/321:00:003
2712999150,10cyclictest0-21swapper/323:42:053
2712899151,12cyclictest4365-21nscd22:55:462
2712899151,12cyclictest11381-21aten2.4_r3power21:10:132
27128991510,3cyclictest0-21swapper/221:29:162
2712899150,3cyclictest0-21swapper/223:30:292
2712899150,13cyclictest0-21swapper/222:01:452
2712799151,2cyclictest28687-21cpuspeed_turbos23:10:221
27127991512,2cyclictest0-21swapper/121:40:191
27127991512,2cyclictest0-21swapper/121:40:181
27127991510,3cyclictest6267-21diskmemload00:28:381
27127991510,3cyclictest0-21swapper/122:40:291
2712799150,2cyclictest0-21swapper/100:30:581
2712699151,8cyclictest12917-21gpgv22:20:000
2712699151,3cyclictest29235-21date19:15:010
27126991512,2cyclictest29875-21kworker/u8:022:42:040
27126991512,2cyclictest26126-21kworker/u8:222:00:050
2712699150,3cyclictest0-21swapper/023:44:190
2712699150,13cyclictest0-21swapper/000:15:280
2712699150,13cyclictest0-21swapper/000:03:000
64282140,0sleep10-21swapper/122:45:511
2712999149,3cyclictest5252-21ssh23:53:443
27129991411,2cyclictest8891-21kworker/u8:019:53:053
27129991411,2cyclictest8891-21kworker/u8:019:47:053
27129991411,2cyclictest8249-21kworker/u8:121:47:043
27129991411,2cyclictest6151-21kworker/u8:200:30:063
27129991411,2cyclictest32146-21kworker/u8:123:59:053
27129991411,2cyclictest2537-21kworker/u8:221:51:053
27129991411,2cyclictest2537-21kworker/u8:221:51:053
27129991411,2cyclictest24676-21kworker/u8:223:11:053
27129991411,2cyclictest24676-21kworker/u8:223:06:023
27129991411,2cyclictest24676-21kworker/u8:222:45:053
27129991411,2cyclictest0-21swapper/319:55:243
2712999140,12cyclictest0-21swapper/300:17:203
27128991411,2cyclictest0-21swapper/223:21:552
27128991411,2cyclictest0-21swapper/223:06:552
27128991411,2cyclictest0-21swapper/222:49:152
27128991411,2cyclictest0-21swapper/200:21:052
2712899140,2cyclictest0-21swapper/223:01:252
2712899140,2cyclictest0-21swapper/200:25:442
2712899140,2cyclictest0-21swapper/200:03:052
2712899140,13cyclictest0-21swapper/222:35:052
2712899140,12cyclictest8446-21ssh23:56:062
2712899140,12cyclictest0-21swapper/223:15:152
2712899140,12cyclictest0-21swapper/222:16:272
2712899140,12cyclictest0-21swapper/219:10:172
2712799149,3cyclictest21864-21hddtemp_smartct20:15:181
27127991410,2cyclictest0-21swapper/119:20:151
2712799140,2cyclictest10994-21diskmemload21:38:061
2712799140,2cyclictest0-21swapper/122:56:491
2712799140,2cyclictest0-21swapper/122:10:491
2712799140,13cyclictest0-21swapper/100:07:581
2712799140,12cyclictest10994-21diskmemload23:30:181
2712799140,12cyclictest0-21swapper/123:05:241
2712799140,12cyclictest0-21swapper/121:45:181
2712799140,12cyclictest0-21swapper/100:20:471
2712699142,7cyclictest1410-21ssh23:50:010
2712699142,7cyclictest0-21swapper/023:09:590
27126991411,2cyclictest28438-21kworker/u8:121:30:010
27126991411,2cyclictest0-21swapper/000:30:180
2712699140,2cyclictest10994-21diskmemload23:55:180
2712699140,2cyclictest0-21swapper/021:25:290
2712699140,2cyclictest0-21swapper/019:15:180
2712699140,12cyclictest0-21swapper/023:12:300
2712699140,12cyclictest0-21swapper/023:00:160
110222140,2sleep00-21swapper/021:43:340
110222140,2sleep00-21swapper/021:43:340
319892130,2sleep00-21swapper/023:47:200
27129991310,2cyclictest972-21kworker/u8:019:32:053
27129991310,2cyclictest5543-21kworker/u8:023:31:053
27129991310,2cyclictest29832-21kworker/u8:219:40:053
27129991310,2cyclictest29125-21kworker/u8:020:37:053
27129991310,2cyclictest26873-21kworker/u8:119:21:053
27129991310,2cyclictest25030-21kworker/u8:200:20:053
27129991310,2cyclictest21558-21kworker/u8:221:26:013
27129991310,2cyclictest0-21swapper/323:37:293
27129991310,2cyclictest0-21swapper/321:00:163
27129991310,2cyclictest0-21swapper/320:15:233
27129991310,2cyclictest0-21swapper/320:00:273
2712999130,2cyclictest0-21swapper/323:16:363
2712999130,2cyclictest0-21swapper/321:05:213
2712999130,2cyclictest0-21swapper/320:10:223
2712999130,11cyclictest0-21swapper/322:52:323
2712999130,11cyclictest0-21swapper/319:15:253
2712899139,2cyclictest0-21swapper/200:07:162
2712899138,3cyclictest22824-21sed21:55:222
2712899138,3cyclictest22824-21sed21:55:212
27128991310,2cyclictest10994-21diskmemload22:25:182
27128991310,2cyclictest10994-21diskmemload22:20:062
27128991310,2cyclictest0-21swapper/222:43:272
2712899130,2cyclictest0-21swapper/223:48:462
2712899130,2cyclictest0-21swapper/221:51:452
2712899130,2cyclictest0-21swapper/221:51:452
2712899130,2cyclictest0-21swapper/221:39:172
2712899130,2cyclictest0-21swapper/221:19:472
2712899130,2cyclictest0-21swapper/200:30:062
2712899130,12cyclictest0-21swapper/223:11:152
2712899130,12cyclictest0-21swapper/222:05:252
2712899130,11cyclictest10994-21diskmemload22:31:182
2712899130,11cyclictest10632-21ssh23:25:152
2712899130,11cyclictest0-21swapper/200:35:212
27127991310,2cyclictest10994-21diskmemload22:36:041
27127991310,2cyclictest10994-21diskmemload22:20:231
27127991310,2cyclictest0-21swapper/123:48:251
27127991310,2cyclictest0-21swapper/123:38:451
27127991310,2cyclictest0-21swapper/122:54:351
27127991310,2cyclictest0-21swapper/122:25:361
27127991310,2cyclictest0-21swapper/122:15:311
27127991310,2cyclictest0-21swapper/122:06:201
27127991310,2cyclictest0-21swapper/121:26:031
27127991310,2cyclictest0-21swapper/100:10:271
2712799130,2cyclictest0-21swapper/123:56:061
2712799130,2cyclictest0-21swapper/122:01:441
2712799130,2cyclictest0-21swapper/100:01:591
2712799130,12cyclictest0-21swapper/123:50:041
2712799130,12cyclictest0-21swapper/123:20:541
2712799130,12cyclictest0-21swapper/123:00:271
2712799130,11cyclictest10994-21diskmemload23:26:541
2712799130,11cyclictest10994-21diskmemload21:10:121
2712699138,3cyclictest23327-21basename20:20:130
2712699131,6cyclictest0-21swapper/020:45:000
27126991310,2cyclictest0-21swapper/023:35:230
27126991310,2cyclictest0-21swapper/022:37:190
27126991310,2cyclictest0-21swapper/020:40:070
27126991310,2cyclictest0-21swapper/019:45:150
27126991310,2cyclictest0-21swapper/019:20:190
27126991310,2cyclictest0-21swapper/000:36:030
2712699130,3cyclictest0-21swapper/000:13:580
2712699130,2cyclictest0-21swapper/021:55:130
2712699130,2cyclictest0-21swapper/021:55:130
2712699130,2cyclictest0-21swapper/000:25:170
2712699130,12cyclictest0-21swapper/023:34:580
2712699130,12cyclictest0-21swapper/022:08:180
2712699130,12cyclictest0-21swapper/019:55:240
2712699130,11cyclictest12823-21ssh23:27:010
2712699130,11cyclictest0-21swapper/021:12:070
2712699130,11cyclictest0-21swapper/000:21:200
186802130,2sleep00-21swapper/021:50:400
186802130,2sleep00-21swapper/021:50:390
81812120,2sleep00-21swapper/022:48:400
2712999129,2cyclictest29832-21kworker/u8:219:38:043
2712999129,2cyclictest29125-21kworker/u8:020:51:043
2712999129,2cyclictest23383-21kworker/u8:020:25:043
2712999129,2cyclictest0-21swapper/319:25:143
2712999120,2cyclictest0-21swapper/322:20:323
2712999120,11cyclictest31266-21copy20:40:013
2712999120,11cyclictest0-21swapper/320:45:193
2712999120,11cyclictest0-21swapper/320:20:153
2712899129,2cyclictest0-21swapper/223:50:352
2712899129,2cyclictest0-21swapper/222:53:342
2712899129,2cyclictest0-21swapper/221:31:172
2712899129,2cyclictest0-21swapper/220:30:172
2712899120,2cyclictest0-21swapper/223:35:162
2712899120,2cyclictest0-21swapper/221:45:472
2712899120,2cyclictest0-21swapper/221:22:082
2712899120,2cyclictest0-21swapper/220:40:162
2712899120,11cyclictest21942-21ssh00:10:272
2712899120,11cyclictest0-21swapper/219:15:182
2712799129,2cyclictest0-21swapper/120:30:141
2712799120,2cyclictest0-21swapper/120:35:241
2712799120,2cyclictest0-21swapper/119:55:211
2712799120,11cyclictest0-21swapper/121:56:521
2712799120,11cyclictest0-21swapper/121:56:511
2712799120,11cyclictest0-21swapper/121:50:261
2712799120,11cyclictest0-21swapper/121:50:251
2712799120,11cyclictest0-21swapper/120:05:171
2712799120,11cyclictest0-21swapper/119:50:271
2712799120,11cyclictest0-21swapper/119:40:251
2712799120,10cyclictest0-21swapper/119:25:231
2712699129,2cyclictest5016-21apt-get19:35:130
2712699129,2cyclictest0-21swapper/022:32:590
2712699122,7cyclictest0-21swapper/020:08:580
2712699122,7cyclictest0-21swapper/019:31:590
2712699122,6cyclictest0-21swapper/021:15:260
2712699120,2cyclictest10994-21diskmemload00:08:090
2712699120,2cyclictest0-21swapper/022:51:230
2712699120,1cyclictest0-21swapper/020:55:150
2712699120,11cyclictest0-21swapper/021:46:110
2712699120,11cyclictest0-21swapper/020:00:210
2712899118,2cyclictest0-21swapper/221:40:082
2712899118,2cyclictest0-21swapper/221:40:072
2712899112,5cyclictest0-21swapper/221:00:002
2712899111,6cyclictest0-21swapper/220:50:252
2712899111,5cyclictest0-21swapper/220:00:002
2712899110,2cyclictest0-21swapper/219:20:222
2712799112,6cyclictest0-21swapper/120:59:591
2712799110,9cyclictest6927-21latency_hist19:40:001
2712799110,2cyclictest27125-21cyclictest20:20:201
2712699118,2cyclictest0-21swapper/023:20:440
2712699118,2cyclictest0-21swapper/021:00:190
2712699118,2cyclictest0-21swapper/020:15:230
2712699117,2cyclictest0-21swapper/021:05:160
2712699116,3cyclictest25564-21grep20:25:140
2712699112,6cyclictest0-21swapper/020:39:020
2712699111,7cyclictest0-21swapper/020:14:590
2712699110,3cyclictest0-21swapper/020:50:280
2712699110,2cyclictest0-21swapper/022:55:330
2712999105,3cyclictest4166-21rsyslogd20:07:363
2712999102,6cyclictest0-21swapper/320:32:593
2712899107,2cyclictest0-21swapper/220:45:152
2712899102,6cyclictest0-21swapper/220:10:282
2712899100,2cyclictest0-21swapper/220:56:372
2712799107,2cyclictest0-21swapper/121:00:141
2712799107,2cyclictest0-21swapper/120:25:131
2712799102,5cyclictest0-21swapper/120:12:021
2712799100,8cyclictest0-21swapper/121:05:171
2712799100,8cyclictest0-21swapper/120:00:211
2712799100,8cyclictest0-21swapper/119:30:171
2712799100,8cyclictest0-21swapper/119:10:111
2712699105,3cyclictest7688-21fschecks_time19:40:140
271289994,3cyclictest9978-21load19:45:202
271289994,3cyclictest4560-21ntpd19:40:182
271289994,3cyclictest3106-21tail19:30:172
271289994,3cyclictest2036-21grep19:25:242
271289994,3cyclictest13172-21cat19:54:592
271289992,4cyclictest24341-21pzem_voltage20:20:232
271289991,6cyclictest0-21swapper/220:15:262
271289990,8cyclictest0-21swapper/220:37:452
271289990,8cyclictest0-21swapper/220:25:132
271279996,2cyclictest0-21swapper/120:45:151
271279996,2cyclictest0-21swapper/120:44:591
271279996,2cyclictest0-21swapper/119:15:171
271279990,8cyclictest0-21swapper/120:50:151
271269994,3cyclictest27960-21sed20:30:160
271269990,3cyclictest2722-21cron19:30:010
271289983,3cyclictest9505-21cut21:05:162
271289983,3cyclictest4994-21date19:35:012
271289983,3cyclictest17392-21stty20:05:142
271269983,3cyclictest11218-21dpkg19:50:130
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional