You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-05 - 23:04

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #5, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack5slot3s.osadl.org (updated Sun Jul 05, 2026 12:44:28)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
39342238162,25sleep00-21swapper/007:07:500
39422237162,23sleep20-21swapper/207:07:562
40712236163,24sleep30-21swapper/307:09:383
39412202165,24sleep10-21swapper/107:07:551
1205421850,4sleep2420299cyclictest11:05:172
1486721620,4sleep0419299cyclictest09:59:580
420299105101,3cyclictest0-21swapper/210:17:492
4202998784,2cyclictest0-21swapper/207:36:462
4202998577,7cyclictest0-21swapper/210:05:252
317112540,2sleep10-21swapper/111:50:151
111782520,3sleep211181-21timerwakeupswit12:15:242
94542490,4sleep123-21ksoftirqd/111:00:141
249922470,1sleep30-21swapper/307:55:123
4202993835,2cyclictest0-21swapper/212:29:362
419299301,14cyclictest31863-21munin-run09:25:010
420999295,17cyclictest27870-21kworker/u8:011:16:573
419299294,12cyclictest26844-21awk08:00:000
419299291,13cyclictest14269-21munin-run08:45:010
420299284,22cyclictest0-21swapper/210:27:552
419299285,11cyclictest5490-21awk08:25:020
419299285,11cyclictest10507-21perf09:50:010
419299281,15cyclictest24641-21munin-run07:55:010
419299281,14cyclictest22104-21munin-run11:30:010
419299274,11cyclictest0-21swapper/011:45:010
419299272,11cyclictest31151-21kworker/0:308:19:300
419299271,14cyclictest21489-21munin-run10:15:010
83102260,4sleep121-21rcuc/110:55:251
420299269,3cyclictest0-21swapper/208:20:232
420299260,11cyclictest0-21swapper/211:18:052
420999250,10cyclictest0-21swapper/308:38:013
420299251,8cyclictest0-21swapper/212:22:012
420299250,16cyclictest0-21swapper/212:35:192
419299258,3cyclictest4477-21kworker/0:111:00:270
419299252,11cyclictest20098-21kworker/0:207:45:130
4192992511,3cyclictest18121-21kworker/0:012:36:150
419299251,12cyclictest19011-21kworker/0:010:20:370
419299251,12cyclictest16265-21kworker/0:308:54:370
419299251,10cyclictest20805-21kworker/0:209:00:330
419299251,10cyclictest15883-21kworker/0:212:29:130
420299248,3cyclictest0-21swapper/212:12:032
420299247,3cyclictest9780-21smart_sda09:45:232
4202992410,3cyclictest0-21swapper/211:31:152
420299240,12cyclictest0-21swapper/209:21:442
419699241,18cyclictest0-21swapper/109:39:561
4196992410,12cyclictest1295-21kworker/u8:008:12:211
419299248,2cyclictest17462-21kworker/0:011:17:370
419299245,4cyclictest23515-21kworker/0:110:40:440
419299242,10cyclictest23515-21kworker/0:110:31:050
419299242,10cyclictest18507-21kworker/0:109:14:330
269112240,1sleep10-21swapper/111:40:121
85152230,5sleep230-21rcuc/210:55:262
420999237,6cyclictest0-21swapper/309:41:153
420999231,16cyclictest0-21swapper/308:06:563
420999231,13cyclictest0-21swapper/310:21:013
420299239,4cyclictest0-21swapper/212:08:112
420299238,4cyclictest0-21swapper/210:46:352
420299238,4cyclictest0-21swapper/210:46:352
420299237,3cyclictest0-21swapper/211:25:112
420299231,14cyclictest0-21swapper/208:35:272
420299230,10cyclictest0-21swapper/210:12:372
419699231,9cyclictest0-21swapper/111:14:131
4196992317,3cyclictest28752-21ntp_states09:15:211
419699231,20cyclictest0-21swapper/111:48:001
419699231,13cyclictest0-21swapper/110:37:061
419299239,3cyclictest6033-21kworker/0:109:42:050
419299233,11cyclictest3156-21/usr/sbin/munin12:00:130
419299233,11cyclictest3156-21/usr/sbin/munin12:00:130
419299232,10cyclictest14521-21kworker/0:210:05:370
419299231,9cyclictest23515-21kworker/0:110:35:060
4192992311,3cyclictest11217-21kworker/0:007:30:210
419299231,10cyclictest4864-21kworker/0:212:15:290
419299231,10cyclictest26454-21kworker/0:011:46:490
4192992310,2cyclictest20098-21kworker/0:208:02:090
419299230,11cyclictest25741-21kworker/0:210:25:390
420999221,10cyclictest0-21swapper/311:58:503
420299229,3cyclictest0-21swapper/209:17:322
420299228,3cyclictest0-21swapper/211:37:402
420299226,3cyclictest0-21swapper/207:40:352
420299225,3cyclictest0-21swapper/208:29:312
420299224,3cyclictest0-21swapper/211:23:232
4202992213,3cyclictest0-21swapper/210:03:232
4202992212,4cyclictest0-21swapper/208:02:512
4202992210,3cyclictest0-21swapper/209:27:162
420299220,13cyclictest0-21swapper/209:40:162
419699222,17cyclictest0-21swapper/107:50:131
419699221,6cyclictest4312-21irqstats08:20:171
419699221,6cyclictest0-21swapper/112:24:281
419699221,2cyclictest9502-21cpu07:20:121
419699221,18cyclictest81562sleep109:43:251
419699221,18cyclictest17390-21tr12:30:211
419699221,18cyclictest11682-21df11:05:141
419699221,16cyclictest0-21swapper/107:42:481
419699221,15cyclictest0-21swapper/112:09:091
419699221,13cyclictest0-21swapper/111:37:271
419699221,12cyclictest9227-40dnf07:19:511
419699220,4cyclictest0-21swapper/110:45:571
419699220,4cyclictest0-21swapper/110:45:571
419299229,3cyclictest32368-21kworker/0:210:49:100
419299229,3cyclictest32368-21kworker/0:210:49:090
419299228,3cyclictest32368-21kworker/0:210:51:410
419299228,3cyclictest31946-21kworker/0:211:53:050
419299228,2cyclictest21238-21kworker/0:210:15:370
419299223,10cyclictest7571-21/usr/sbin/munin12:10:190
419299222,10cyclictest26723-21kworker/0:008:05:370
419299221,7cyclictest32722-21sed09:25:160
419299221,3cyclictest0-21swapper/012:23:280
419299221,10cyclictest25127-21kworker/0:009:09:330
419299221,10cyclictest24211-21kworker/0:111:34:130
206862220,1sleep30-21swapper/307:45:133
420999211,6cyclictest0-21swapper/310:05:033
420999211,11cyclictest30399-21cat10:35:013
420999211,11cyclictest10718-21/usr/sbin/munin09:50:123
420999211,11cyclictest0-21swapper/310:11:003
420999210,7cyclictest0-21swapper/311:49:373
420299219,7cyclictest0-21swapper/212:32:592
420299217,3cyclictest0-21swapper/208:43:232
420299215,3cyclictest0-21swapper/211:02:592
420299214,8cyclictest31935-21irqstats11:50:172
420299213,6cyclictest25554-40dnf10:20:592
420299213,3cyclictest0-21swapper/207:19:442
420299211,9cyclictest0-21swapper/209:52:432
420299211,6cyclictest0-21swapper/211:45:112
420299211,11cyclictest0-21swapper/212:00:192
420299211,11cyclictest0-21swapper/212:00:192
420299211,11cyclictest0-21swapper/211:55:242
420299210,13cyclictest0-21swapper/208:53:392
419699218,4cyclictest0-21swapper/109:10:511
419699214,8cyclictest0-21swapper/111:15:161
4196992117,3cyclictest0-21swapper/110:11:561
419699211,6cyclictest0-21swapper/109:55:401
4196992113,2cyclictest0-21swapper/111:25:251
419699211,2cyclictest2024-21if_enp1s008:15:151
419699211,15cyclictest0-21swapper/111:57:551
419699211,11cyclictest15751-21smart_sda08:45:251
419699211,11cyclictest12666-21uniq08:40:161
419699211,11cyclictest0-21swapper/111:25:001
419699211,10cyclictest0-21swapper/108:36:471
419699210,7cyclictest0-21swapper/112:11:291
419699210,18cyclictest27801-21if_enp2s008:00:171
419299218,3cyclictest4477-21kworker/0:111:05:530
419299217,3cyclictest4864-21kworker/0:212:07:010
419299217,2cyclictest26454-21kworker/0:011:55:410
419299216,4cyclictest16265-21kworker/0:308:47:530
419299211,5cyclictest12691-21munin-run09:55:010
4192992115,3cyclictest8914-21grep11:00:010
419299211,11cyclictest17647-21kworker/0:311:22:210
420999209,3cyclictest808-21snmpd07:25:193
420999206,6cyclictest0-21swapper/311:52:193
420999206,4cyclictest0-21swapper/310:45:333
420999206,4cyclictest0-21swapper/310:45:333
420999206,2cyclictest13891-21diskstats11:10:133
420999205,6cyclictest0-21swapper/310:52:173
420999201,5cyclictest1393-21ntp_states10:40:233
420999201,12cyclictest0-21swapper/312:17:373
420999201,12cyclictest0-21swapper/309:04:313
420999200,9cyclictest0-21swapper/310:00:363
420999200,7cyclictest0-21swapper/312:23:593
420299208,3cyclictest0-21swapper/211:42:562
420299207,3cyclictest0-21swapper/211:12:272
420299201,12cyclictest0-21swapper/209:05:112
419699209,3cyclictest15220-21ntp_states12:25:211
419699208,5cyclictest21398-21kworker/u8:210:22:261
419699207,4cyclictest17263-21latency_hist10:05:021
419699206,11cyclictest24373-21awk09:05:221
419699205,12cyclictest17226-21fschecks_time08:50:161
419699202,11cyclictest87550irq/128-enp1s0-11:30:201
4196992016,2cyclictest25263-21diskmemload09:51:361
419699201,2cyclictest10817-21ntpq12:15:201
419699201,17cyclictest0-21swapper/110:07:281
419699201,14cyclictest0-21swapper/107:26:321
419699201,11cyclictest5297-21perl10:50:161
419699201,11cyclictest19956-21ntp_states08:55:211
419699201,11cyclictest0-21swapper/110:17:211
419699200,18cyclictest0-21swapper/110:31:261
419299209,3cyclictest9485-21/usr/sbin/munin07:20:210
419299206,3cyclictest15883-21kworker/0:212:30:370
419299202,2cyclictest17868-21kworker/0:107:40:150
419299201,11cyclictest7284-21/usr/sbin/munin07:15:200
419299201,11cyclictest27690-21/usr/sbin/munin09:15:190
419299201,11cyclictest24528-21/usr/sbin/munin11:35:180
419299201,11cyclictest15108-21/usr/sbin/munin10:00:240
420999199,2cyclictest25601-21ntp_states11:35:233
420999198,3cyclictest30506-21/usr/sbin/munin10:35:183
420999197,3cyclictest808-21snmpd07:37:453
420999196,6cyclictest0-21swapper/311:00:553
420999196,5cyclictest0-21swapper/311:41:283
420999196,5cyclictest0-21swapper/309:32:103
420999194,5cyclictest0-21swapper/310:28:443
420999192,6cyclictest20247-21munin-run07:45:023
420999191,9cyclictest0-21swapper/312:01:153
420999191,9cyclictest0-21swapper/312:01:153
420999191,16cyclictest0-21swapper/309:08:563
420999191,15cyclictest32057-21grep08:10:153
420999191,10cyclictest808-21snmpd12:28:433
420999191,10cyclictest7902-21/usr/sbin/munin08:30:223
420999191,10cyclictest0-21swapper/312:39:413
420999191,10cyclictest0-21swapper/307:14:393
420299196,4cyclictest0-21swapper/209:30:322
420299193,3cyclictest87550irq/128-enp1s0-08:55:532
420299192,3cyclictest0-21swapper/208:10:592
4202991916,2cyclictest0-21swapper/207:46:532
4202991912,4cyclictest30787-21grep08:05:242
420299191,15cyclictest5211-21fschecks_time10:50:162
420299191,10cyclictest0-21swapper/208:17:312
420299190,3cyclictest0-21swapper/207:50:272
420299190,12cyclictest0-21swapper/209:35:382
420299190,12cyclictest0-21swapper/207:34:432
419699197,3cyclictest0-21swapper/112:03:481
419699197,3cyclictest0-21swapper/112:03:481
419699196,3cyclictest23-21ksoftirqd/108:05:391
419699195,1cyclictest0-21swapper/107:55:141
419699192,6cyclictest21899-21irqstats09:00:201
419699191,17cyclictest0-21swapper/107:45:301
419699191,14cyclictest0-21swapper/107:11:441
4196991911,3cyclictest0-21swapper/107:32:251
419699191,12cyclictest0-21swapper/110:42:331
419699191,10cyclictest30859-21needreboot09:20:171
419699191,10cyclictest0-21swapper/109:25:101
419299195,3cyclictest4477-21kworker/0:111:10:050
419299193,2cyclictest87450irq/127-enp1s0-08:38:510
61642181,2sleep00-21swapper/007:12:410
420999183,3cyclictest0-21swapper/309:20:483
420999182,4cyclictest12409-21ntp_states11:05:213
420999182,3cyclictest0-21swapper/309:12:313
420999181,9cyclictest0-21swapper/308:51:233
420999181,7cyclictest3397-21date08:20:013
420999181,4cyclictest5368-21/usr/sbin/munin12:05:263
420999181,4cyclictest29322-21sshd09:15:513
420999181,11cyclictest0-21swapper/312:12:063
420999181,11cyclictest0-21swapper/311:23:473
420999181,11cyclictest0-21swapper/309:57:083
420999181,10cyclictest0-21swapper/311:25:143
420999181,10cyclictest0-21swapper/310:55:023
420999181,10cyclictest0-21swapper/309:28:483
420299182,3cyclictest0-21swapper/210:34:312
420299181,5cyclictest0-21swapper/209:56:072
420299181,3cyclictest0-21swapper/209:03:152
4202991812,3cyclictest14825-21df08:45:132
4202991810,3cyclictest0-21swapper/210:44:392
419699185,3cyclictest0-21swapper/110:26:101
419699182,3cyclictest9501-21vmstat08:30:271
419699181,7cyclictest3048-21pmu-power09:30:231
419699181,16cyclictest0-21swapper/109:47:111
419699181,16cyclictest0-21swapper/108:25:511
419699181,16cyclictest0-21swapper/107:38:191
419699181,15cyclictest0-21swapper/112:35:521
419299188,2cyclictest87550irq/128-enp1s0-08:30:350
419299186,3cyclictest31151-21kworker/0:308:11:000
419299181,7cyclictest0-21swapper/009:36:130
35932181,2sleep00-21swapper/009:31:210
76042170,2sleep00-21swapper/008:29:170
420999177,3cyclictest0-21swapper/312:30:213
420999176,3cyclictest808-21snmpd08:42:573
420999173,4cyclictest4680-21wc08:20:223
420999173,2cyclictest0-21swapper/311:30:403
420999172,9cyclictest41-21ksoftirqd/307:17:463
420999171,6cyclictest27930-21latency08:00:183
420999171,6cyclictest23241-21perl07:50:133
420999171,6cyclictest0-21swapper/307:20:263
420999171,3cyclictest0-21swapper/307:30:273
4209991711,3cyclictest9464-21missed_timers09:45:183
420999171,10cyclictest0-21swapper/310:18:033
420999171,10cyclictest0-21swapper/309:35:423
420299174,3cyclictest19255-21kworker/2:008:32:412
420299173,5cyclictest25990-21pmu-power07:55:222
420299171,3cyclictest0-21swapper/207:13:032
420299171,14cyclictest10747-21smart_sda07:20:242
420299170,11cyclictest0-21swapper/210:36:352
419299170,15cyclictest20873-21awk09:00:010
177102171,2sleep00-21swapper/007:35:550
420999161,7cyclictest0-21swapper/308:27:563
420999161,6cyclictest18888-21/usr/sbin/munin08:55:253
420999161,6cyclictest14492-21/usr/sbin/munin08:45:163
420299161,3cyclictest0-21swapper/209:10:492
420299161,3cyclictest0-21swapper/207:25:072
4192991611,3cyclictest12492-21uname07:25:170
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional