You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2024-04-18 - 18:50

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #c, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackcslot7s.osadl.org (updated Thu Apr 18, 2024 12:44:55)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
124086521200,16sleep10-21swapper/107:10:021
1241025991052,37cyclictest1311507-21sshd10:28:492
12410219910410,91cyclictest0-21swapper/109:33:191
1623991010,10rtkit-daemon1753-21gnome-shell11:35:461
1623991000,34rtkit-daemon881634-21Timer11:59:411
13495132980,5chrt0-21swapper/212:15:292
13495132980,5chrt0-21swapper/212:15:282
124102599983,89cyclictest880726-21firefox07:44:082
1241021999812,14cyclictest881251-21Softwar~cThread10:26:201
124102799972,90cyclictest880726-21firefox09:25:543
1241025999684,8cyclictest1753-21gnome-shell09:33:202
124102199962,89cyclictest1267319-21smartctl08:20:371
124069929656,9sleep00-21swapper/007:08:410
1241025999582,8cyclictest881251-21Softwar~cThread09:25:532
124102199952,6cyclictest0-21swapper/109:09:141
124048229574,14sleep20-21swapper/207:05:532
124102199942,88cyclictest0-21swapper/110:04:411
124102199942,7cyclictest0-21swapper/108:18:181
1241021999420,50cyclictest881251-21Softwar~cThread07:49:261
124102799932,87cyclictest0-21swapper/307:29:343
124102799932,6cyclictest0-21swapper/309:57:523
124102599932,6cyclictest0-21swapper/212:29:422
124102199932,87cyclictest0-21swapper/109:25:181
124102799923,84cyclictest1753-21gnome-shell12:18:193
124102799923,84cyclictest1753-21gnome-shell12:18:193
124102799922,6cyclictest0-21swapper/309:33:563
124102799922,6cyclictest0-21swapper/307:39:383
124102599922,87cyclictest0-21swapper/208:59:322
124102599922,86cyclictest1246630-21chrt07:24:072
124102599922,86cyclictest0-21swapper/209:56:492
124102199922,87cyclictest0-21swapper/109:03:111
124102199922,86cyclictest0-21swapper/109:51:531
124102199922,86cyclictest0-21swapper/109:51:531
124102199922,86cyclictest0-21swapper/108:29:361
124101999922,88cyclictest0-21swapper/009:22:330
124101999922,86cyclictest0-21swapper/007:50:200
1241027999183,5cyclictest0-21swapper/309:03:113
124102799912,85cyclictest0-21swapper/308:18:183
124102599912,87cyclictest0-21swapper/208:23:022
124102199912,85cyclictest0-21swapper/112:18:191
124102199912,85cyclictest0-21swapper/112:18:191
124102199911,86cyclictest231irq_work/110:15:011
1241027999083,4cyclictest0-21swapper/310:04:403
1241027999082,5cyclictest0-21swapper/309:51:543
1241027999082,5cyclictest0-21swapper/309:51:533
1241027999081,5cyclictest0-21swapper/310:15:023
1241025999082,5cyclictest0-21swapper/207:29:352
124102599902,85cyclictest0-21swapper/210:10:042
124102599901,57cyclictest1753-21gnome-shell12:37:302
1241025999010,19cyclictest881251-21Softwar~cThread07:45:192
1241019999083,5cyclictest0-21swapper/012:05:570
13339002890,5sleep30-21swapper/311:30:293
1241027998981,5cyclictest0-21swapper/309:07:593
124102599892,6cyclictest0-21swapper/210:21:262
12722612880,19sleep126-21ksoftirqd/108:35:221
124049328869,13sleep30-21swapper/307:06:023
1241025998779,5cyclictest0-21swapper/208:18:192
124102199872,19cyclictest1753-21gnome-shell09:48:031
1241025998677,6cyclictest0-21swapper/209:09:152
124102599862,64cyclictest0-21swapper/212:09:562
1241025998577,5cyclictest0-21swapper/209:24:252
124102199859,49cyclictest1306817-21expr10:15:141
124102199852,19cyclictest1753-21gnome-shell12:39:471
1241027998476,5cyclictest0-21swapper/308:53:083
1241027998475,5cyclictest0-21swapper/312:39:493
124102799842,22cyclictest1753-21gnome-shell12:21:483
124102799842,17cyclictest1315730-21/usr/sbin/munin10:40:133
124102799842,17cyclictest1315730-21/usr/sbin/munin10:40:133
1241027998412,66cyclictest1617-10pipewire-media-09:10:323
124102799841,21cyclictest1753-21gnome-shell10:23:533
124102599849,69cyclictest881251-21Softwar~cThread12:10:062
124102599842,77cyclictest1248992-21expr07:30:122
124102599842,77cyclictest1248992-21expr07:30:122
1241021998412,12cyclictest881251-21Softwar~cThread08:55:201
1241019998476,5cyclictest0-21swapper/012:32:250
162399830,8rtkit-daemon1622-21rtkit-daemon10:26:523
124102799832,20cyclictest1753-21gnome-shell09:43:293
1241025998375,5cyclictest0-21swapper/209:12:182
124102199837,71cyclictest881251-21Softwar~cThread09:43:461
124102199832,19cyclictest1753-21gnome-shell07:38:081
1241019998376,5cyclictest0-21swapper/010:14:060
12871642820,4sleep20-21swapper/209:19:552
124102799822,20cyclictest1753-21gnome-shell08:20:423
124102799822,20cyclictest1753-21gnome-shell07:41:503
124102799821,20cyclictest1753-21gnome-shell11:38:023
124102599822,74cyclictest1753-21gnome-shell11:34:392
124102599821,76cyclictest1753-21gnome-shell10:19:542
124102199826,51cyclictest141rcu_preempt10:41:191
124102199826,51cyclictest141rcu_preempt10:41:191
124102799812,20cyclictest1753-21gnome-shell07:30:133
124102799812,20cyclictest1753-21gnome-shell07:30:133
124102799812,19cyclictest1753-21gnome-shell10:36:033
124102799812,18cyclictest1753-21gnome-shell08:55:123
124102799812,11cyclictest860997-21snmpd12:29:563
124102599812,73cyclictest1301309-21/usr/sbin/munin10:00:242
124102599812,37cyclictest1753-21gnome-shell09:00:052
1241025998114,63cyclictest1069-21runrttasks08:51:202
124102199812,18cyclictest1753-21gnome-shell12:04:101
1241019998174,5cyclictest0-21swapper/008:49:260
124102799802,19cyclictest1753-21gnome-shell07:22:003
124102799802,10cyclictest880981-21Cache23
1241025998072,5cyclictest0-21swapper/211:36:372
124102599802,72cyclictest1753-21gnome-shell10:05:472
124102199801,19cyclictest1753-21gnome-shell09:17:251
124102799799,11cyclictest885448-21Timer10:34:043
124102799792,10cyclictest1255435-21seq07:47:313
1241027997920,54cyclictest1753-21gnome-shell12:33:163
1241025997971,5cyclictest0-21swapper/208:29:362
162399780,8rtkit-daemon921-21thermald12:03:090
124102799782,15cyclictest860997-21snmpd08:26:423
124102799781,9cyclictest1287016-21chrt09:18:023
124102799781,17cyclictest1753-21gnome-shell09:35:253
1241025997870,5cyclictest0-21swapper/209:51:542
1241025997870,5cyclictest0-21swapper/209:51:532
124102599782,35cyclictest1753-21gnome-shell12:21:552
1241025997815,22cyclictest34-21ksoftirqd/212:34:072
1241025997812,45cyclictest0-21swapper/210:35:162
124102199782,26cyclictest881251-21Softwar~cThread09:39:171
124102199781,51cyclictest1252453-21cstates07:40:121
1241019997871,5cyclictest0-21swapper/008:27:140
13321632770,5sleep30-21swapper/311:25:303
13321632770,5sleep30-21swapper/311:25:303
124102799772,12cyclictest1753-21gnome-shell10:18:063
124102799772,11cyclictest1753-21gnome-shell12:06:093
124102799772,11cyclictest1753-21gnome-shell07:50:323
124102599772,69cyclictest1753-21gnome-shell10:30:142
124102599771,35cyclictest1753-21gnome-shell07:35:402
1241021997757,8cyclictest1753-21gnome-shell12:32:371
124102199772,10cyclictest13152432chrt10:38:351
1241019997726,46cyclictest881251-21Softwar~cThread07:27:070
162399760,8rtkit-daemon921-21thermald12:21:060
124102799762,20cyclictest1753-21gnome-shell10:05:483
124102799762,13cyclictest1753-21gnome-shell09:20:133
124102599762,68cyclictest1753-21gnome-shell07:18:282
124102599762,30cyclictest1753-21gnome-shell10:40:242
124102599762,30cyclictest1753-21gnome-shell10:40:232
124102199763,11cyclictest1753-21gnome-shell07:32:061
124102199763,11cyclictest1753-21gnome-shell07:32:051
124102199761,52cyclictest1346508-21taskset12:07:081
1241019997668,5cyclictest0-21swapper/012:29:420
1241019997617,54cyclictest1314170-21cat10:35:170
124102799752,67cyclictest1753-21gnome-shell08:49:553
124102799752,11cyclictest1753-21gnome-shell08:30:133
124102599752,67cyclictest1753-21gnome-shell09:45:342
1241021997547,5cyclictest141rcu_preempt10:21:011
124102199752,47cyclictest1753-21gnome-shell12:20:241
124102199752,10cyclictest880726-21firefox12:27:491
1241019997567,5cyclictest0-21swapper/007:24:060
1241019997517,15cyclictest1252331-21cat07:40:110
124102799742,15cyclictest1753-21gnome-shell07:18:203
124102599742,67cyclictest0-21swapper/209:40:132
124102199743,47cyclictest880726-21firefox09:13:061
124102199742,10cyclictest1278543-21munin-run08:55:001
124102799734,7cyclictest141rcu_preempt09:45:323
124102599732,66cyclictest1753-21gnome-shell09:35:382
1241025997313,37cyclictest881251-21Softwar~cThread08:01:132
1241025997312,38cyclictest881251-21Softwar~cThread08:37:072
124101999739,17cyclictest1753-21gnome-shell09:40:010
124101999736,8cyclictest141rcu_preempt11:35:530
1241019997366,5cyclictest0-21swapper/010:28:260
1241019997315,53cyclictest438-20systemd-journal12:40:000
124102799721,10cyclictest1753-21gnome-shell12:04:203
1241025997212,38cyclictest881251-21Softwar~cThread08:32:492
1241019997265,5cyclictest0-21swapper/010:21:260
1241019997219,49cyclictest1302352-21timerwakeupswit10:00:340
124102599712,63cyclictest1753-21gnome-shell12:05:022
124102599712,63cyclictest1753-21gnome-shell07:50:032
124102199713,48cyclictest880726-21firefox07:24:041
124102199713,47cyclictest1300352-21ntp_states09:55:271
1241021997112,15cyclictest881251-21Softwar~cThread11:33:531
1241019997163,5cyclictest0-21swapper/009:57:420
1241019997163,5cyclictest0-21swapper/008:59:330
1241019997116,51cyclictest1294642-21seq09:40:130
124102199703,47cyclictest1753-21gnome-shell07:26:311
124102199702,45cyclictest1287815-21expr09:20:141
1241019997063,4cyclictest0-21swapper/008:23:020
124102599692,62cyclictest1753-21gnome-shell08:50:002
124102199693,45cyclictest1753-21gnome-shell12:10:301
1241019996910,16cyclictest881251-21Softwar~cThread07:18:210
124102199683,45cyclictest1304547-21munin-run10:10:001
124101999689,16cyclictest1753-21gnome-shell10:16:040
124101999687,19cyclictest1753-21gnome-shell09:07:100
124101999683,47cyclictest0-21swapper/008:18:570
1241021996715,48cyclictest1617-10pipewire-media-08:50:011
124101999679,16cyclictest1753-21gnome-shell09:17:380
124101999672,22cyclictest1753-21gnome-shell09:52:130
124101999672,22cyclictest1753-21gnome-shell09:52:130
124101999669,53cyclictest1753-21gnome-shell12:13:470
124101999669,53cyclictest1753-21gnome-shell10:34:040
124101999667,16cyclictest1753-21gnome-shell09:01:080
124101999662,21cyclictest1753-21gnome-shell09:31:540
124101999662,19cyclictest1753-21gnome-shell10:09:000
13408822650,8chrt1753-21gnome-shell11:50:253
124102599652,42cyclictest1753-21gnome-shell08:41:312
124102599652,42cyclictest1753-21gnome-shell08:41:302
124102599652,34cyclictest1753-21gnome-shell11:48:042
124101999658,53cyclictest1753-21gnome-shell07:47:340
124101999658,53cyclictest1753-21gnome-shell07:39:480
124101999658,52cyclictest881251-21Softwar~cThread07:33:110
124101999658,52cyclictest881251-21Softwar~cThread07:33:100
1241019996548,12cyclictest881369-21Timer09:27:050
124101999652,21cyclictest1753-21gnome-shell09:13:590
124101999652,21cyclictest1753-21gnome-shell08:53:380
1241019996513,48cyclictest1753-21gnome-shell09:49:400
124102599641,39cyclictest860997-21snmpd11:50:052
1241021996448,11cyclictest881251-21Softwar~cThread07:18:331
1241021996437,22cyclictest1753-21gnome-shell07:50:241
124102199642,56cyclictest1753-21gnome-shell10:30:031
124101999647,53cyclictest1753-21gnome-shell10:40:180
124101999647,53cyclictest1753-21gnome-shell10:40:180
124101999646,15cyclictest1753-21gnome-shell11:34:220
124101999642,19cyclictest1753-21gnome-shell12:16:370
124101999642,19cyclictest1753-21gnome-shell12:16:370
124102599622,34cyclictest1753-21gnome-shell08:05:062
124102599622,32cyclictest1753-21gnome-shell11:00:412
124102599612,33cyclictest1753-21gnome-shell11:16:472
124102599612,29cyclictest1753-21gnome-shell11:24:082
124102599602,33cyclictest880726-21firefox11:59:352
124102599602,32cyclictest1753-21gnome-shell11:12:392
124102599602,31cyclictest1753-21gnome-shell10:51:512
124102599602,31cyclictest1753-21gnome-shell07:57:022
1241021996011,44cyclictest26-21ksoftirqd/108:30:101
1241027995923,31cyclictest1343089-21sshd11:57:513
124101999592,53cyclictest0-21swapper/008:30:220
124102599582,32cyclictest1753-21gnome-shell07:12:332
124102599582,31cyclictest1753-21gnome-shell10:58:242
124102599562,29cyclictest1753-21gnome-shell08:10:142
162399550,9rtkit-daemon1622-21rtkit-daemon11:51:561
1241027995420,29cyclictest1260869-21/usr/sbin/munin08:05:003
1241027995320,29cyclictest1258068-21ls07:55:183
1241025995325,24cyclictest1325212-21cat11:05:302
1241025995325,24cyclictest1325212-21cat11:05:292
1241025995312,36cyclictest881251-21Softwar~cThread11:26:002
1241025995312,36cyclictest881251-21Softwar~cThread11:25:592
1241027995212,34cyclictest881338-21IPC3
13364542510,4sleep20-21swapper/211:40:102
124102199519,16cyclictest1753-21gnome-shell11:40:261
1241021995118,7cyclictest1325986-21latency_hist11:10:011
1241021995118,7cyclictest1325986-21latency_hist11:10:011
1241021995115,10cyclictest1261486-21df08:05:141
1241021995113,12cyclictest1320960-21ls10:55:141
1241021995112,12cyclictest1317565-21grep10:45:141
1241027995020,26cyclictest1273724-21cut08:40:133
1241027995020,26cyclictest1273724-21cut08:40:133
1241027995019,26cyclictest994-21lldpd11:21:243
1241027995017,28cyclictest881251-21Softwar~cThread11:19:393
1241027995016,30cyclictest1242204-21grep07:10:313
1241021995021,6cyclictest26-21ksoftirqd/107:58:521
1241021995018,11cyclictest1327507-21cron11:15:011
1241021995015,8cyclictest1241689-21awk07:10:191
1241021995015,14cyclictest1319220-21cat10:50:111
124102799493,42cyclictest1263721-21grep08:10:303
1241027994910,35cyclictest1338803-40lsb_release11:45:233
1241025994917,27cyclictest1318477-21seq10:45:342
1241021994917,12cyclictest880966-21IPC1
1241021994917,12cyclictest880966-21IPC1
13195402480,4chrt0-21swapper/010:50:170
1241027994819,24cyclictest1753-21gnome-shell10:51:093
1241021994816,8cyclictest1263530-21head08:10:251
1241021994811,12cyclictest1273683-21cat08:40:141
1241021994811,12cyclictest1273683-21cat08:40:131
124101999488,12cyclictest1753-21gnome-shell11:45:350
1241019994841,5cyclictest161rcuc/011:28:090
1241019994841,5cyclictest161rcuc/011:28:090
1241027994714,28cyclictest1326260-21cstates11:10:123
1241027994713,29cyclictest1322951-21sh11:00:163
124101999475,13cyclictest13-21ksoftirqd/007:55:100
1241027994618,23cyclictest1069-21runrttasks10:47:303
1241027994616,25cyclictest1324528-21cstates11:05:123
1241027994616,25cyclictest1324528-21cstates11:05:123
124102199469,15cyclictest1753-21gnome-shell11:45:191
1241021994616,12cyclictest231irq_work/111:18:451
1241021994614,11cyclictest1259476-21cpuspeed_turbos08:00:131
124102799452,10cyclictest1753-21gnome-shell10:55:353
1241027994517,24cyclictest1272588-21ntp_states08:35:283
124101999457,21cyclictest1753-21gnome-shell11:05:550
124101999457,21cyclictest1753-21gnome-shell11:05:550
124101999455,13cyclictest0-21swapper/011:40:120
124101999455,11cyclictest13-21ksoftirqd/008:03:090
124101999454,19cyclictest13-21ksoftirqd/008:36:400
124102799441,13cyclictest1338041-21sh11:45:023
124101999443,16cyclictest1753-21gnome-shell11:10:270
124101999438,13cyclictest1358-1kworker/u9:111:16:190
124101999434,9cyclictest13-21ksoftirqd/007:10:350
1241021994214,7cyclictest1330484-21timerandwakeup11:20:321
124101999429,19cyclictest13-21ksoftirqd/010:56:580
124101999426,16cyclictest1340659-21memory11:50:200
124101999426,12cyclictest1753-21gnome-shell08:15:000
1241019994233,7cyclictest13-21ksoftirqd/011:24:400
124102199411,9cyclictest1753-21gnome-shell11:04:451
124101999419,5cyclictest0-21swapper/008:05:320
124101999412,11cyclictest1317920-21sed10:45:210
1241019994112,8cyclictest1358-1kworker/u9:111:56:010
1241019994010,13cyclictest0-21swapper/011:00:440
124101999394,6cyclictest0-21swapper/008:45:010
124101999394,6cyclictest0-21swapper/008:45:010
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional