You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-05-13 - 02:45

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #c, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackcslot7s.osadl.org (updated Mon May 12, 2025 12:44:56)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
75856022840,8sleep00-21swapper/007:10:010
77775521030,16sleep342-21ksoftirqd/308:00:273
75864899952,89cyclictest2197014-21PK-Backend11:03:020
75864899952,6cyclictest2197014-21PK-Backend12:03:250
75864899952,6cyclictest2197014-21PK-Backend11:22:540
8426042940,7sleep32197014-21PK-Backend11:45:143
75865699942,88cyclictest835148-21runrttasks11:21:533
75864899942,87cyclictest2197014-21PK-Backend10:32:500
75864899942,87cyclictest2197014-21PK-Backend10:03:560
75864899942,6cyclictest2197014-21PK-Backend11:38:230
758656999384,6cyclictest0-21swapper/312:10:423
75865699932,87cyclictest0-21swapper/312:09:073
75864899932,87cyclictest2197014-21PK-Backend12:07:530
75864899932,86cyclictest2197014-21PK-Backend11:17:100
75864899932,6cyclictest2197014-21PK-Backend11:11:320
75833229347,37sleep30-21swapper/307:07:563
8290572920,7sleep10-21swapper/110:55:171
758656999284,5cyclictest0-21swapper/310:34:043
75864899922,85cyclictest2197014-21PK-Backend09:57:510
758656998981,5cyclictest0-21swapper/311:39:043
758656998981,5cyclictest0-21swapper/310:05:103
758656998880,5cyclictest0-21swapper/311:52:453
758656998880,5cyclictest0-21swapper/311:52:453
758656998880,5cyclictest0-21swapper/311:05:353
758652998828,19cyclictest34-21ksoftirqd/211:45:242
758656998780,4cyclictest2197014-21PK-Backend11:32:123
758656998778,5cyclictest0-21swapper/310:25:153
758656998678,5cyclictest0-21swapper/311:14:233
75864899861,21cyclictest1753-21gnome-shell10:53:320
75864899861,20cyclictest1753-21gnome-shell11:40:370
758656998577,5cyclictest0-21swapper/311:01:343
75865699849,70cyclictest1753-21gnome-shell10:49:413
758656998476,5cyclictest0-21swapper/311:43:323
758656998476,5cyclictest0-21swapper/310:02:153
75864899841,21cyclictest1753-21gnome-shell10:18:500
75864899841,19cyclictest1753-21gnome-shell10:22:360
75864899832,17cyclictest1753-21gnome-shell12:14:330
162399830,10rtkit-daemon1753-21gnome-shell09:22:201
758656998274,5cyclictest0-21swapper/311:58:023
758656998274,5cyclictest0-21swapper/310:13:263
75865699822,74cyclictest1753-21gnome-shell10:15:123
758648998210,67cyclictest1775-21gvfs-afc-volume10:45:030
758656998174,4cyclictest0-21swapper/309:59:003
758656998173,5cyclictest0-21swapper/310:51:403
75865699811,38cyclictest1753-21gnome-shell12:00:143
75864899812,19cyclictest1753-21gnome-shell11:25:390
75864899812,19cyclictest1753-21gnome-shell11:25:390
75864899811,19cyclictest1753-21gnome-shell11:57:250
75865699802,35cyclictest1753-21gnome-shell10:24:283
75864899802,19cyclictest1753-21gnome-shell10:07:040
75864899801,18cyclictest1753-21gnome-shell11:33:030
75865699791,72cyclictest1753-21gnome-shell10:43:433
75865699791,72cyclictest1753-21gnome-shell10:43:433
75864899792,18cyclictest1753-21gnome-shell10:40:260
75864899792,18cyclictest1753-21gnome-shell10:40:250
75864899791,19cyclictest1753-21gnome-shell10:55:250
75864899789,14cyclictest141rcu_preempt11:53:250
75864899789,14cyclictest141rcu_preempt11:53:250
75864899781,18cyclictest1753-21gnome-shell10:11:170
162399780,8rtkit-daemon1753-21gnome-shell08:31:021
162399780,5rtkit-daemon0-21swapper/112:11:161
758656997667,6cyclictest0-21swapper/310:37:433
75865699762,70cyclictest1753-21gnome-shell10:57:063
75865699761,70cyclictest1753-21gnome-shell11:16:203
758652997665,7cyclictest1753-21gnome-shell10:37:492
75864899761,10cyclictest1753-21gnome-shell10:30:000
75865699752,69cyclictest1753-21gnome-shell11:29:093
75865699752,69cyclictest1753-21gnome-shell11:29:093
75822627548,17sleep10-21swapper/107:06:331
162399750,5rtkit-daemon0-21swapper/110:43:271
162399750,5rtkit-daemon0-21swapper/110:43:271
162399750,5rtkit-daemon0-21swapper/007:42:050
75865699742,67cyclictest853813-21processes12:15:303
758652997463,7cyclictest1753-21gnome-shell09:57:132
75864899742,10cyclictest1753-21gnome-shell10:35:220
162399740,5rtkit-daemon0-21swapper/111:57:301
162399740,5rtkit-daemon0-21swapper/109:41:261
758652997316,15cyclictest2962348-21rs:main2
758650997326,6cyclictest0-21swapper/111:04:571
75865099732,49cyclictest821775-21/usr/sbin/munin10:15:221
162399730,6rtkit-daemon2197014-21PK-Backend09:46:180
162399730,6rtkit-daemon2197014-21PK-Backend09:46:180
162399730,6rtkit-daemon2197014-21PK-Backend08:16:290
162399730,6rtkit-daemon2197014-21PK-Backend07:45:370
162399730,5rtkit-daemon2197014-21PK-Backend08:02:190
75864899722,9cyclictest853383-21hddtemp_smartct12:15:210
75864899722,10cyclictest831322-21perf11:10:000
758652997160,7cyclictest1753-21gnome-shell10:43:382
758652997160,7cyclictest1753-21gnome-shell10:43:382
758652997160,7cyclictest1753-21gnome-shell10:20:362
162399710,6rtkit-daemon2197014-21PK-Backend11:49:590
162399710,52rtkit-daemon26-21ksoftirqd/109:11:241
758652997018,47cyclictest34-21ksoftirqd/210:48:172
75865099701,50cyclictest26-21ksoftirqd/112:15:321
162399700,6rtkit-daemon2197014-21PK-Backend07:28:150
75865699697,34cyclictest798606-21gunzip09:00:283
758650996961,5cyclictest26-21ksoftirqd/111:20:221
758652996857,7cyclictest1753-21gnome-shell10:26:122
758652996854,10cyclictest34-21ksoftirqd/210:32:162
162399680,46rtkit-daemon831955-21munin-run11:15:011
758652996656,6cyclictest1753-21gnome-shell10:51:302
75865299652,59cyclictest1753-21gnome-shell11:51:192
75865299652,59cyclictest1753-21gnome-shell11:51:192
75865299652,22cyclictest1753-21gnome-shell10:03:222
758650996551,9cyclictest1753-21gnome-shell11:32:501
758650996551,9cyclictest1753-21gnome-shell11:09:141
758650996551,9cyclictest1753-21gnome-shell10:35:191
75865099652,59cyclictest1753-21gnome-shell10:27:251
75837626550,9sleep20-21swapper/207:08:312
75865299642,57cyclictest1753-21gnome-shell11:02:022
75865299641,21cyclictest1753-21gnome-shell10:20:002
75865099642,19cyclictest1753-21gnome-shell11:45:341
758652996340,17cyclictest1753-21gnome-shell11:06:102
75865299632,57cyclictest1753-21gnome-shell11:32:052
75865299632,56cyclictest1753-21gnome-shell12:15:022
75865299632,56cyclictest1753-21gnome-shell12:07:262
75865299632,56cyclictest1753-21gnome-shell11:35:582
75865299632,21cyclictest1753-21gnome-shell11:20:182
75865299631,57cyclictest1753-21gnome-shell11:29:262
75865299631,57cyclictest1753-21gnome-shell11:29:262
75865099638,13cyclictest836529-21processes11:25:321
75865099638,13cyclictest836529-21processes11:25:321
75865099632,57cyclictest1753-21gnome-shell12:09:471
75865099632,57cyclictest1753-21gnome-shell10:45:591
75865099632,57cyclictest0-21swapper/111:38:441
75865099631,19cyclictest1753-21gnome-shell11:15:291
75865699622,43cyclictest1753-21gnome-shell08:12:403
75865299622,56cyclictest1753-21gnome-shell12:04:152
75865299622,56cyclictest1753-21gnome-shell11:58:422
75865299622,55cyclictest1753-21gnome-shell11:10:302
75865299622,54cyclictest1753-21gnome-shell11:15:102
758650996247,9cyclictest1753-21gnome-shell12:00:021
75865099622,6cyclictest2197014-21PK-Backend11:40:201
75865099622,57cyclictest0-21swapper/111:51:491
75865099622,57cyclictest0-21swapper/111:51:481
75865099622,18cyclictest1753-21gnome-shell10:55:031
75865299612,54cyclictest1753-21gnome-shell11:42:332
75865099611,56cyclictest1753-21gnome-shell10:33:351
75865099611,56cyclictest1753-21gnome-shell10:09:041
75865299602,17cyclictest1753-21gnome-shell10:55:222
75865099602,55cyclictest0-21swapper/110:04:021
75865699592,33cyclictest1753-21gnome-shell09:05:213
75865699592,33cyclictest1753-21gnome-shell09:05:213
75865699592,30cyclictest1753-21gnome-shell07:40:173
75865699592,25cyclictest0-21swapper/307:20:153
75865699591,31cyclictest1-21systemd09:35:013
75865699591,29cyclictest855306-21missed_timers12:20:223
75865299591,54cyclictest1753-21gnome-shell10:14:022
75865099592,15cyclictest822964-21runrttasks10:23:071
162399590,9rtkit-daemon1622-21rtkit-daemon08:53:163
162399590,38rtkit-daemon0-21swapper/107:21:161
7941712580,6chrt0-21swapper/308:50:013
75865699582,29cyclictest854-21avahi-daemon09:25:173
75865699582,29cyclictest854-21avahi-daemon09:25:173
75865699582,29cyclictest1753-21gnome-shell09:11:563
162399580,37rtkit-daemon0-21swapper/109:19:551
75865699572,32cyclictest796603-21memory08:55:243
75865699572,32cyclictest1753-21gnome-shell08:40:183
75865699572,30cyclictest1753-21gnome-shell07:59:373
75865699572,28cyclictest1753-21gnome-shell07:16:413
75865699562,37cyclictest1753-21gnome-shell09:20:283
75865699562,30cyclictest859041-21python312:30:273
75865699561,30cyclictest1753-21gnome-shell08:16:203
75865099562,51cyclictest818662-21runrttasks09:57:361
75865099562,12cyclictest1753-21gnome-shell10:10:431
758648995623,28cyclictest768004-21cron07:35:000
75865699552,37cyclictest1753-21gnome-shell07:27:343
75865699552,36cyclictest1753-21gnome-shell08:35:023
75865699552,29cyclictest1753-21gnome-shell12:25:403
75865699552,28cyclictest1753-21gnome-shell08:25:003
758648995525,25cyclictest797878-21tr09:00:140
758648995520,14cyclictest816764-40apt-check09:51:180
75865699542,37cyclictest1753-21gnome-shell09:52:093
75865699542,36cyclictest1753-21gnome-shell09:48:053
75865699542,36cyclictest1753-21gnome-shell09:48:053
75865699542,29cyclictest1753-21gnome-shell07:35:383
75865699542,28cyclictest1753-21gnome-shell08:33:023
75865699542,28cyclictest1753-21gnome-shell07:48:293
758648995420,11cyclictest438-20systemd-journal09:34:350
758648995419,15cyclictest812637-21ntpq09:40:240
758648995417,8cyclictest8043702sleep009:15:590
758648995310,6cyclictest181irq_work/008:05:350
75865699522,28cyclictest766511-21apt07:30:123
75865699522,27cyclictest1753-21gnome-shell09:40:223
758652995231,8cyclictest141rcu_preempt12:15:242
758652995227,20cyclictest1753-21gnome-shell08:20:172
758648995218,9cyclictest1753-21gnome-shell07:13:310
75865699512,27cyclictest1753-21gnome-shell09:18:263
75865699512,25cyclictest1753-21gnome-shell08:08:273
758650995128,18cyclictest817293-21latency_hist09:55:011
75865699502,26cyclictest1753-21gnome-shell08:25:513
75865699502,26cyclictest1753-21gnome-shell08:25:513
75864899501,23cyclictest1753-21gnome-shell08:44:310
758648995012,12cyclictest800994-21latency_hist09:10:020
758648995012,12cyclictest800994-21latency_hist09:10:020
758648995011,13cyclictest801812-21cat09:10:230
75865699492,28cyclictest774274-21ntp_states07:50:283
75864899498,16cyclictest1753-21gnome-shell07:55:540
758648994916,10cyclictest13-21ksoftirqd/009:21:200
758648994911,14cyclictest860781-21swap12:35:300
75865699482,25cyclictest1753-21gnome-shell12:39:373
758656994816,27cyclictest759086-21expr07:10:153
758648994817,8cyclictest792768-21cstates08:45:140
758648994715,10cyclictest13-21ksoftirqd/008:56:250
758652994623,18cyclictest760341-21munin-node07:15:092
75864899469,18cyclictest1753-21gnome-shell07:38:350
75864899469,16cyclictest1753-21gnome-shell12:20:380
75864899469,16cyclictest1753-21gnome-shell09:36:070
75864899469,16cyclictest1753-21gnome-shell08:38:440
758648994618,7cyclictest2197014-21PK-Backend12:27:170
758648994617,8cyclictest786721-21users08:25:330
758648994617,8cyclictest786721-21users08:25:320
758648994614,10cyclictest13-21ksoftirqd/009:29:210
758648994614,10cyclictest13-21ksoftirqd/009:29:210
758656994514,27cyclictest811674-21cut09:40:023
758652994534,8cyclictest1753-21gnome-shell07:45:202
75864899451,23cyclictest1753-21gnome-shell08:52:350
758652994420,20cyclictest1753-21gnome-shell08:00:232
75865099433,28cyclictest0-21swapper/109:35:591
758650994213,24cyclictest795030-21ntp_states08:50:261
758650994213,16cyclictest26-21ksoftirqd/108:15:331
758652994118,19cyclictest1753-21gnome-shell08:55:002
758652994117,20cyclictest1753-21gnome-shell08:35:262
75865099412,10cyclictest3242254-21gmain09:50:021
75865099412,10cyclictest3242254-21gmain09:50:011
75864899411,5cyclictest2197014-21PK-Backend07:53:060
162399410,28rtkit-daemon0-21swapper/008:20:320
758650994030,7cyclictest1753-21gnome-shell08:06:461
758650994029,7cyclictest1753-21gnome-shell08:36:111
758650994021,14cyclictest438-20systemd-journal07:30:281
758650994018,7cyclictest764728-21cstates07:25:161
758652993923,12cyclictest855-21dbus-daemon08:11:562
758652993913,5cyclictest141rcu_preempt09:50:222
758650993929,7cyclictest26-21ksoftirqd/112:30:021
758650993929,7cyclictest1753-21gnome-shell09:28:141
758650993929,7cyclictest1753-21gnome-shell09:28:131
758650993925,10cyclictest771525-21perf07:45:011
758650993923,12cyclictest781523-21uptime08:10:331
758650993922,13cyclictest761432-21diskstats07:15:171
758650993920,16cyclictest251rcuc/108:03:341
758650993919,17cyclictest251rcuc/107:38:301
75865299382,8cyclictest1753-21gnome-shell09:11:422
758652993822,12cyclictest786379-21ntpq08:25:252
758652993822,12cyclictest786379-21ntpq08:25:252
758652993820,13cyclictest994-21lldpd07:26:072
75865299381,33cyclictest1753-21gnome-shell12:21:102
758650993828,7cyclictest1753-21gnome-shell09:01:501
758650993823,11cyclictest783871-21seq08:20:041
758650993821,13cyclictest808800-21df_inode09:30:161
758648993819,14cyclictest858540-21df_inode12:30:150
162399380,26rtkit-daemon0-21swapper/112:21:091
758652993725,8cyclictest1753-21gnome-shell09:23:352
758652993721,11cyclictest856911-21cat12:25:212
758650993723,10cyclictest760167-21sshd07:13:331
162399370,26rtkit-daemon2197014-21PK-Backend07:22:120
75865299362,7cyclictest1753-21gnome-shell09:50:002
75865299362,7cyclictest1753-21gnome-shell09:50:002
758652993627,6cyclictest2197014-21PK-Backend12:40:002
758652993619,13cyclictest788721-21seq08:33:482
758652993618,14cyclictest806995-21expr09:25:152
758652993618,14cyclictest806995-21expr09:25:152
75865299361,11cyclictest859-21NetworkManager08:19:412
75865299361,11cyclictest769812-21idleruntime-cro07:40:012
758650993626,7cyclictest1753-21gnome-shell12:33:201
758650993624,8cyclictest799652-21tr09:05:171
758650993624,8cyclictest799652-21tr09:05:161
758650993621,11cyclictest860469-21cat12:35:221
758652993517,13cyclictest994-21lldpd07:43:082
758652993516,15cyclictest855-21dbus-daemon12:31:562
758652993510,21cyclictest803997-31apt-check09:15:302
758650993526,5cyclictest26-21ksoftirqd/107:47:331
758650993524,8cyclictest1753-21gnome-shell08:45:271
758650993524,7cyclictest1753-21gnome-shell08:55:521
7921472342,6sleep21753-21gnome-shell08:43:252
75865299345,10cyclictest34-21ksoftirqd/209:00:392
75865299343,11cyclictest775237-21/usr/sbin/munin07:55:192
758652993421,9cyclictest34-21ksoftirqd/208:55:162
75865299341,13cyclictest809629-21taskset09:31:502
758650993424,7cyclictest1753-21gnome-shell07:51:561
758650993416,5cyclictest26-21ksoftirqd/107:55:331
758648993423,7cyclictest1069-21runrttasks08:10:460
758648993418,12cyclictest761212-21cut07:15:130
75864899341,28cyclictest2197014-21PK-Backend08:30:080
75865299331,9cyclictest2197014-21PK-Backend09:37:022
75865299331,9cyclictest2197014-21PK-Backend07:12:102
758652993316,13cyclictest1-21systemd08:10:012
758650993315,13cyclictest26-21ksoftirqd/108:25:071
758650993315,13cyclictest26-21ksoftirqd/108:25:061
758652993222,5cyclictest34-21ksoftirqd/209:05:302
758652993222,5cyclictest34-21ksoftirqd/209:05:302
758652993220,8cyclictest1825-21JS14:31:442
758652993217,11cyclictest763864-21uptime07:20:322
758652993216,12cyclictest793326-21awk08:45:272
758652993210,18cyclictest1753-21gnome-shell09:44:552
758650993220,9cyclictest26-21ksoftirqd/108:45:011
75865299311,12cyclictest767256-40lsb_release07:30:272
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional