You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2024-04-26 - 10:12

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #c, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackcslot7s.osadl.org (updated Fri Apr 26, 2024 00:44:58)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
127752322340,8sleep30-21swapper/319:05:243
127718621140,10sleep11753-21gnome-shell19:05:181
1623991120,5rtkit-daemon0-21swapper/120:56:400
1623991120,5rtkit-daemon0-21swapper/120:56:400
127834999992,86cyclictest1331065-21sshd21:38:500
127834999987,87cyclictest181irq_work/023:55:120
127834999962,8cyclictest0-21swapper/020:39:010
127834999952,90cyclictest0-21swapper/020:24:170
13666452940,5sleep0161rcuc/023:20:150
127834999942,88cyclictest0-21swapper/019:12:040
127834999942,7cyclictest0-21swapper/023:10:370
127834999942,7cyclictest0-21swapper/020:51:520
127834999942,7cyclictest0-21swapper/000:10:540
127834999942,7cyclictest0-21swapper/000:10:540
127834999932,88cyclictest0-21swapper/000:24:560
127834999932,7cyclictest0-21swapper/020:49:580
127834999932,6cyclictest0-21swapper/023:41:450
127834999932,6cyclictest0-21swapper/020:10:560
127834999932,6cyclictest0-21swapper/019:53:530
127834999932,6cyclictest0-21swapper/000:04:050
127834999931,6cyclictest0-21swapper/021:34:390
1278364999243,8cyclictest1753-21gnome-shell00:00:283
1278349999284,5cyclictest0-21swapper/021:18:320
127834999922,6cyclictest0-21swapper/021:11:580
162399910,78rtkit-daemon0-21swapper/019:05:540
127834999913,84cyclictest0-21swapper/020:18:250
162399900,11rtkit-daemon1622-21rtkit-daemon21:18:443
127836099903,6cyclictest0-21swapper/220:55:512
127836099903,6cyclictest0-21swapper/220:55:512
1278352999030,10cyclictest1753-21gnome-shell20:51:191
127804229055,28sleep20-21swapper/219:07:532
1278360998982,4cyclictest0-21swapper/220:52:172
1278349998911,16cyclictest881251-21Softwar~cThread21:40:100
1278349998822,48cyclictest1370637-21processes23:30:290
1278360998562,14cyclictest34-21ksoftirqd/219:18:592
1278360998562,14cyclictest34-21ksoftirqd/219:18:592
127836099852,11cyclictest0-21swapper/220:40:022
127836099849,50cyclictest1289925-21cat19:40:212
127836099849,50cyclictest1289925-21cat19:40:212
127836099842,10cyclictest0-21swapper/200:38:052
1278364998325,16cyclictest1753-21gnome-shell23:21:133
1278352998326,45cyclictest26-21ksoftirqd/121:00:301
127834999832,20cyclictest0-21swapper/019:40:200
127834999832,20cyclictest0-21swapper/019:40:190
127836099822,12cyclictest0-21swapper/223:34:482
1278352998226,12cyclictest881251-21Softwar~cThread23:41:401
1278360998172,5cyclictest1327836-21latency_hist21:30:012
1278360998152,11cyclictest34-21ksoftirqd/221:20:542
1278360998152,11cyclictest34-21ksoftirqd/221:20:542
1278349998163,13cyclictest881251-21Softwar~cThread20:01:050
127834999812,19cyclictest1753-21gnome-shell23:38:320
127836499802,20cyclictest1753-21gnome-shell00:14:553
127836499802,20cyclictest1753-21gnome-shell00:14:553
127836499802,19cyclictest1753-21gnome-shell20:36:123
127836099802,19cyclictest1753-21gnome-shell00:12:262
127836099802,19cyclictest1753-21gnome-shell00:12:262
127836099802,12cyclictest1753-21gnome-shell19:31:072
127834999802,61cyclictest1295127-21cstates19:55:140
127834999802,19cyclictest1753-21gnome-shell19:31:570
127834999802,19cyclictest0-21swapper/023:25:300
127836499795,12cyclictest0-21swapper/319:37:533
127836099792,52cyclictest1753-21gnome-shell00:16:462
127836099792,48cyclictest1391612-21switchtime00:30:322
127836099791,50cyclictest1753-21gnome-shell00:00:432
127836099791,19cyclictest1753-21gnome-shell20:28:362
127836099791,19cyclictest1753-21gnome-shell20:22:352
1278360997911,11cyclictest141rcu_preempt20:32:582
1278360997911,11cyclictest141rcu_preempt20:32:572
127834999793,12cyclictest1335287-21df_inode21:50:160
127834999792,13cyclictest1311820-40lsb_release20:42:200
127834999792,12cyclictest1753-21gnome-shell21:27:170
1278360997839,14cyclictest1368830-21taskset23:25:252
1278360997826,10cyclictest881251-21Softwar~cThread19:37:412
127836099782,49cyclictest1753-21gnome-shell19:25:102
127836099782,13cyclictest1753-21gnome-shell19:57:302
127836099782,13cyclictest1753-21gnome-shell00:28:152
127836099781,19cyclictest1753-21gnome-shell21:48:442
127836099781,19cyclictest1753-21gnome-shell21:48:432
1278352997826,47cyclictest881251-21Softwar~cThread23:31:551
127834999783,58cyclictest1333576-21hddtemp_smartct21:45:220
127834999783,58cyclictest1333576-21hddtemp_smartct21:45:220
127834999782,15cyclictest0-21swapper/000:30:140
127836099779,14cyclictest141rcu_preempt19:14:442
127836099777,9cyclictest141rcu_preempt21:30:162
127836099773,45cyclictest1294261-40lsb_release19:51:252
127836099772,49cyclictest1753-21gnome-shell19:49:432
127834999773,11cyclictest1292704-21/usr/sbin/munin19:50:000
127836499761,71cyclictest1753-21gnome-shell19:33:343
127836099763,9cyclictest1364862-21expr23:15:122
1278360997625,46cyclictest881251-21Softwar~cThread21:15:302
127836099762,44cyclictest1753-21gnome-shell21:09:192
127836099762,13cyclictest1753-21gnome-shell23:40:442
127836099762,12cyclictest1388168-21threads00:20:302
127836099762,11cyclictest1753-21gnome-shell23:05:142
1278352997661,10cyclictest1375380-21expr23:45:161
127835299765,7cyclictest141rcu_preempt21:35:261
127834999765,57cyclictest141rcu_preempt23:18:260
127834999763,58cyclictest1753-21gnome-shell23:45:590
127834999762,53cyclictest1361549-21hddtemp_smartct23:05:200
127834999762,47cyclictest1753-21gnome-shell21:24:440
127834999762,47cyclictest1753-21gnome-shell21:24:440
127834999762,20cyclictest1383482-21munin-plugin-st00:10:000
127834999762,20cyclictest0-21swapper/020:28:130
127834999762,17cyclictest0-21swapper/000:25:320
127834999762,15cyclictest1753-21gnome-shell20:33:570
127834999762,15cyclictest1753-21gnome-shell20:33:570
1278364997528,44cyclictest1753-21gnome-shell19:51:273
1278364997516,55cyclictest1753-21gnome-shell21:36:013
1278364997516,55cyclictest1753-21gnome-shell21:34:003
1278364997511,51cyclictest0-21swapper/323:46:263
127836099753,10cyclictest1753-21gnome-shell20:05:032
127836099753,10cyclictest1753-21gnome-shell20:05:032
127836099752,46cyclictest1753-21gnome-shell21:38:262
127836099752,45cyclictest1753-21gnome-shell21:03:252
127836099752,13cyclictest1753-21gnome-shell23:37:022
127836099752,12cyclictest1753-21gnome-shell20:15:072
127834999755,11cyclictest1385444-21taskset00:15:030
127834999752,56cyclictest1393000-21latency00:35:220
127834999752,48cyclictest1753-21gnome-shell21:05:150
1278349997517,53cyclictest881251-21Softwar~cThread21:04:520
1278364997425,8cyclictest1753-21gnome-shell00:31:593
1278364997412,7cyclictest141rcu_preempt19:55:253
127836099743,11cyclictest1753-21gnome-shell23:46:382
127836099743,11cyclictest1753-21gnome-shell20:03:102
127836099742,6cyclictest0-21swapper/221:11:522
127836099742,45cyclictest880726-21firefox21:40:552
127836099742,13cyclictest1753-21gnome-shell23:52:362
127836099742,13cyclictest1753-21gnome-shell23:10:592
127836099742,12cyclictest1753-21gnome-shell00:05:282
127836099742,10cyclictest1753-21gnome-shell19:24:282
127834999742,54cyclictest1284623-21df_inode19:25:170
127834999742,17cyclictest0-21swapper/023:50:120
127834999742,11cyclictest1753-21gnome-shell19:15:130
127834999742,11cyclictest1753-21gnome-shell19:15:120
1278364997348,19cyclictest42-21ksoftirqd/323:15:243
1278364997322,47cyclictest1753-21gnome-shell19:13:203
127836099732,48cyclictest1753-21gnome-shell23:23:022
127836099732,48cyclictest1753-21gnome-shell20:38:202
127836099732,12cyclictest1753-21gnome-shell20:13:172
127836099732,11cyclictest1753-21gnome-shell21:50:172
127835299732,15cyclictest141rcu_preempt19:25:091
127835299732,14cyclictest141rcu_preempt23:50:511
127834999738,7cyclictest141rcu_preempt19:35:500
127834999732,14cyclictest1298352-21cpu20:05:140
127834999732,14cyclictest1298352-21cpu20:05:140
127834999732,13cyclictest1282884-21/usr/sbin/munin19:20:150
162399720,9rtkit-daemon1622-21rtkit-daemon22:44:383
162399720,5rtkit-daemon0-21swapper/120:56:341
162399720,5rtkit-daemon0-21swapper/120:56:341
1278364997225,6cyclictest1244849-21kworker/3:023:34:433
127836499722,29cyclictest1753-21gnome-shell19:26:153
127836499722,10cyclictest1753-21gnome-shell20:21:373
127836099722,15cyclictest141rcu_preempt23:56:012
127835299723,14cyclictest141rcu_preempt19:15:211
127835299723,14cyclictest141rcu_preempt19:15:211
127835299722,15cyclictest141rcu_preempt19:51:271
1278352997210,6cyclictest141rcu_preempt21:20:241
1278352997210,6cyclictest141rcu_preempt21:20:241
127834999722,53cyclictest0-21swapper/023:04:390
1278364997125,7cyclictest0-21swapper/300:28:563
127836499712,11cyclictest1753-21gnome-shell00:39:483
1278364997110,7cyclictest141rcu_preempt20:15:193
127836099712,45cyclictest1753-21gnome-shell23:04:482
127835299712,15cyclictest141rcu_preempt19:35:131
1278364997049,6cyclictest141rcu_preempt23:44:363
127836499702,48cyclictest1753-21gnome-shell20:27:493
1278364997011,43cyclictest141rcu_preempt21:40:253
127835299702,15cyclictest141rcu_preempt23:26:421
1278352997015,18cyclictest881251-21Softwar~cThread21:05:121
127835299701,14cyclictest141rcu_preempt20:20:311
1278352997010,7cyclictest141rcu_preempt21:16:551
1278352997010,6cyclictest141rcu_preempt19:42:421
1278352997010,6cyclictest141rcu_preempt19:42:421
1278364996946,17cyclictest42-21ksoftirqd/321:20:303
1278364996946,17cyclictest42-21ksoftirqd/321:20:303
127836499692,63cyclictest1753-21gnome-shell21:08:053
127836499692,46cyclictest1753-21gnome-shell23:35:093
127835299699,7cyclictest141rcu_preempt21:31:131
1278352996959,6cyclictest1308713-21/usr/sbin/munin20:35:251
1278352996917,10cyclictest1331412-21/usr/sbin/munin21:40:311
1278364996852,8cyclictest42-21ksoftirqd/320:33:253
1278364996852,8cyclictest42-21ksoftirqd/320:33:253
1278364996839,25cyclictest1753-21gnome-shell20:05:363
1278364996839,25cyclictest1753-21gnome-shell20:05:353
1278364996822,40cyclictest42-21ksoftirqd/320:55:223
1278364996822,40cyclictest42-21ksoftirqd/320:55:223
1278352996840,8cyclictest26-21ksoftirqd/100:05:001
127835299682,23cyclictest1753-21gnome-shell19:20:271
1278352996815,49cyclictest1360848-21sshd23:04:501
1278352996812,51cyclictest881251-21Softwar~cThread20:11:581
1278364996752,8cyclictest42-21ksoftirqd/321:10:303
1278364996751,10cyclictest1753-21gnome-shell19:24:263
127835299678,6cyclictest141rcu_preempt20:15:461
127835299672,15cyclictest141rcu_preempt00:31:561
1278364996651,9cyclictest1753-21gnome-shell20:02:193
1278364996651,8cyclictest42-21ksoftirqd/323:55:343
1278364996651,10cyclictest1753-21gnome-shell00:15:573
1278364996651,10cyclictest1753-21gnome-shell00:06:023
1278364996641,19cyclictest42-21ksoftirqd/319:15:103
1278364996641,19cyclictest42-21ksoftirqd/319:15:103
127836499662,6cyclictest141rcu_preempt20:40:103
127836499662,60cyclictest1753-21gnome-shell19:41:423
127836499662,60cyclictest1753-21gnome-shell19:41:423
1278352996649,12cyclictest880970-21Timer20:26:011
1278352996616,45cyclictest1361724-21load23:05:231
1278352996615,41cyclictest26-21ksoftirqd/119:48:051
1278364996550,10cyclictest1753-21gnome-shell21:48:113
1278364996550,10cyclictest1753-21gnome-shell21:48:113
1278364996550,10cyclictest1753-21gnome-shell21:04:593
1278364996550,10cyclictest1753-21gnome-shell20:12:313
1278364996549,10cyclictest1753-21gnome-shell19:47:243
1278364996522,6cyclictest42-21ksoftirqd/320:50:143
127836099651,7cyclictest141rcu_preempt20:50:012
127835299652,19cyclictest1753-21gnome-shell20:03:291
127835299652,10cyclictest1753-21gnome-shell00:15:011
127835299652,10cyclictest1753-21gnome-shell00:15:011
1278352996515,9cyclictest1367094-21ntp_offset23:20:291
1278352996515,47cyclictest26-21ksoftirqd/100:24:461
1278364996451,8cyclictest42-21ksoftirqd/300:20:323
1278364996448,10cyclictest1753-21gnome-shell23:50:303
1278364996447,11cyclictest1753-21gnome-shell23:29:043
127836499642,58cyclictest1753-21gnome-shell23:06:533
127836499642,20cyclictest1753-21gnome-shell21:25:393
127835299642,7cyclictest141rcu_preempt23:10:311
127835299642,57cyclictest1753-21gnome-shell23:35:281
1278352996425,6cyclictest141rcu_preempt21:50:001
1278352996425,6cyclictest141rcu_preempt21:50:001
127835299642,19cyclictest1753-21gnome-shell19:34:391
127835299641,6cyclictest141rcu_preempt23:15:171
1278364996319,39cyclictest42-21ksoftirqd/323:10:193
127835299635,6cyclictest141rcu_preempt20:30:171
127835299635,6cyclictest141rcu_preempt20:30:171
1278352996326,8cyclictest26-21ksoftirqd/123:55:321
127835299632,20cyclictest1753-21gnome-shell19:10:091
1278352996315,7cyclictest26-21ksoftirqd/100:07:331
1278352996314,8cyclictest26-21ksoftirqd/100:18:091
1278352996313,7cyclictest26-21ksoftirqd/120:06:541
1278352996313,7cyclictest26-21ksoftirqd/120:06:541
127836499622,19cyclictest1753-21gnome-shell21:50:043
127835299622,56cyclictest1753-21gnome-shell21:25:311
1278352996214,7cyclictest26-21ksoftirqd/120:48:271
162399610,9rtkit-daemon1622-21rtkit-daemon22:59:152
162399610,8rtkit-daemon1622-21rtkit-daemon22:20:262
1278364996147,7cyclictest42-21ksoftirqd/320:50:023
127835299612,4cyclictest141rcu_preempt21:50:301
1278352996114,7cyclictest26-21ksoftirqd/100:37:261
1278352996039,5cyclictest141rcu_preempt00:26:231
1278349996038,15cyclictest1753-21gnome-shell22:18:260
127836499592,53cyclictest1753-21gnome-shell23:04:473
127836099592,53cyclictest881454-21TaskCon~ller2
127835299592,14cyclictest1295642-21proc_pri19:55:301
162399580,9rtkit-daemon1622-21rtkit-daemon22:47:433
162399580,9rtkit-daemon1622-21rtkit-daemon22:05:083
162399580,8rtkit-daemon1622-21rtkit-daemon21:55:202
127835299583,13cyclictest1753-21gnome-shell21:11:551
162399570,8rtkit-daemon1622-21rtkit-daemon21:59:253
162399560,8rtkit-daemon1622-21rtkit-daemon22:51:133
162399560,8rtkit-daemon1622-21rtkit-daemon22:36:593
162399560,8rtkit-daemon1622-21rtkit-daemon22:36:593
127836099560,51cyclictest1622-21rtkit-daemon22:07:232
162399550,8rtkit-daemon1622-21rtkit-daemon22:11:483
127835299543,47cyclictest1310199-21apt-get20:40:041
162399530,8rtkit-daemon1622-21rtkit-daemon22:52:232
127834999502,9cyclictest1753-21gnome-shell22:30:230
127834999502,9cyclictest1753-21gnome-shell22:30:230
1278352994939,7cyclictest1263491-21kworker/1:122:18:261
127836499472,42cyclictest0-21swapper/322:18:263
127836099449,19cyclictest34-21ksoftirqd/222:35:012
127836099449,19cyclictest34-21ksoftirqd/222:35:012
13505092430,11sleep31350503-21apt-config22:35:023
13505092430,11sleep31350503-21apt-config22:35:023
1278364994223,7cyclictest42-21ksoftirqd/322:25:303
1278364994022,13cyclictest42-21ksoftirqd/322:05:033
1278349994022,13cyclictest438-20systemd-journal22:24:500
1278352993926,9cyclictest1348571-21systemd22:30:011
1278352993913,21cyclictest881251-21Softwar~cThread22:24:501
1278352993911,22cyclictest1753-21gnome-shell22:35:061
1278352993911,22cyclictest1753-21gnome-shell22:35:061
1278364993822,5cyclictest42-21ksoftirqd/322:57:293
1278364993819,6cyclictest42-21ksoftirqd/322:20:223
1278352993820,13cyclictest1342008-21cut22:10:111
127834999382,6cyclictest0-21swapper/022:13:030
1278349993821,13cyclictest1348155-21users22:25:340
162399370,29rtkit-daemon1506-21systemd22:44:032
1278360993720,13cyclictest1347850-21awk22:25:292
1278352993720,12cyclictest438-20systemd-journal22:59:551
1278352993717,16cyclictest1355077-21smartctl22:45:341
1278352993717,15cyclictest881251-21Softwar~cThread22:06:551
1278352993713,19cyclictest1348782-21acpi22:30:121
1278352993713,19cyclictest1348782-21acpi22:30:121
1278352993712,13cyclictest881251-21Softwar~cThread22:54:051
127834999372,9cyclictest1753-21gnome-shell22:44:440
1278349993725,8cyclictest1753-21gnome-shell22:53:040
1278360993618,13cyclictest880970-21Timer22:35:262
1278360993618,13cyclictest880970-21Timer22:35:262
1278360993618,13cyclictest1343356-21systemd-run22:15:012
1278352993617,14cyclictest438-20systemd-journal22:40:021
1278352993616,17cyclictest1338557-21cut22:00:121
1278349993619,12cyclictest2962348-21rs:main0
1278349993619,12cyclictest2962348-21rs:main0
1278349993617,14cyclictest881331-21Privileged14:31:440
1278360993519,11cyclictest1338539-21cut22:00:122
127834999352,5cyclictest0-21swapper/022:47:450
1278349993519,12cyclictest1358771-21awk22:55:340
127834999351,31cyclictest0-21swapper/022:09:180
1278352993411,7cyclictest1336855-21cstates21:55:121
1278360993318,11cyclictest34-21ksoftirqd/222:48:132
1278349993317,12cyclictest1338362-21basename22:00:090
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional