You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2025-12-14 - 22:28

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #c, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rackcslot7s.osadl.org (updated Sun Dec 14, 2025 12:44:55)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
100823022080,9sleep10-21swapper/107:08:011
197399990,79rtkit-daemon0-21swapper/207:07:592
10999172920,5sleep0161rcuc/011:25:190
10685072810,3sleep30-21swapper/309:55:203
10482062800,5sleep20-21swapper/209:00:012
100853399764,15cyclictest141rcu_preempt07:27:580
100854299752,68cyclictest2069-21gnome-shell08:07:582
100854599742,52cyclictest2069-21gnome-shell09:43:593
100854299742,67cyclictest2069-21gnome-shell10:37:592
100854299742,67cyclictest2069-21gnome-shell09:52:592
10864312730,6sleep10-21swapper/110:45:281
11216352720,16chrt42-21ksoftirqd/312:25:303
100853399725,8cyclictest141rcu_preempt11:17:590
100764627243,21sleep30-21swapper/307:05:193
100853399706,36cyclictest2069-21gnome-shell08:44:580
10751862690,7chrt4497-21PK-Backend10:15:141
10686742690,4sleep10-21swapper/109:55:221
100854299692,62cyclictest2069-21gnome-shell09:49:592
100853399692,7cyclictest141rcu_preempt12:20:590
100854599682,46cyclictest2069-21gnome-shell07:17:583
100853399683,7cyclictest141rcu_preempt11:24:590
100853399683,7cyclictest141rcu_preempt08:57:580
10912152670,5sleep2331rcuc/211:00:182
100854299672,60cyclictest2069-21gnome-shell09:11:582
100854299671,61cyclictest2069-21gnome-shell08:15:582
100853399672,14cyclictest141rcu_preempt10:47:590
100853399671,42cyclictest4497-21PK-Backend12:13:590
100853399671,42cyclictest4497-21PK-Backend12:13:590
100854599662,5cyclictest0-21swapper/308:20:583
100854299662,61cyclictest0-21swapper/209:40:592
100854299662,60cyclictest2069-21gnome-shell10:28:592
100854299662,24cyclictest0-21swapper/209:56:592
100854299662,24cyclictest0-21swapper/209:08:582
100854599652,44cyclictest141rcu_preempt11:09:593
100854599652,42cyclictest0-21swapper/309:03:583
100854299652,59cyclictest0-21swapper/210:00:592
100854299652,59cyclictest0-21swapper/210:00:592
100854299652,59cyclictest0-21swapper/209:01:582
100854599642,59cyclictest0-21swapper/309:51:593
100854299642,59cyclictest0-21swapper/208:13:582
1008545996338,21cyclictest2069-21gnome-shell10:39:593
100854299632,58cyclictest0-21swapper/210:40:592
100738926342,13sleep00-21swapper/007:05:160
1008545996238,20cyclictest2069-21gnome-shell08:04:583
1008545996238,20cyclictest2069-21gnome-shell08:04:583
100854299622,57cyclictest0-21swapper/207:16:582
100853399622,53cyclictest2069-21gnome-shell07:40:580
11207992610,13sleep234-21ksoftirqd/212:25:132
100854599612,42cyclictest0-21swapper/311:27:593
1008545996036,20cyclictest2069-21gnome-shell09:09:583
100854599602,17cyclictest2069-21gnome-shell07:14:583
100854599602,17cyclictest2069-21gnome-shell07:14:583
1008545996020,35cyclictest1031309-21/usr/sbin/munin08:10:273
100854299602,54cyclictest0-21swapper/208:53:582
1008545995938,19cyclictest0-21swapper/310:29:593
1008545995930,25cyclictest2069-21gnome-shell10:47:593
100854599592,53cyclictest0-21swapper/307:28:583
100854599591,33cyclictest2069-21gnome-shell10:15:593
100854299592,55cyclictest0-21swapper/211:17:592
100854299592,54cyclictest0-21swapper/212:21:592
1008533995910,8cyclictest1056878-21kworker/0:211:39:590
197399580,8rtkit-daemon1972-21rtkit-daemon08:53:090
100854599582,52cyclictest2069-21gnome-shell09:39:593
100854599582,42cyclictest0-21swapper/307:24:583
100854299582,53cyclictest0-21swapper/211:20:592
1008537995843,10cyclictest2069-21gnome-shell07:33:581
1008545995738,14cyclictest2069-21gnome-shell09:18:593
100854599572,50cyclictest2069-21gnome-shell07:55:583
100854599571,30cyclictest2069-21gnome-shell09:27:593
100854299572,29cyclictest1009475-21ntp_states07:10:202
100854299572,29cyclictest1009475-21ntp_states07:10:202
100854299571,51cyclictest1054272-21netstat09:15:232
100853799572,14cyclictest2069-21gnome-shell11:34:591
100853799572,14cyclictest2069-21gnome-shell11:34:581
1008545995638,15cyclictest0-21swapper/312:15:593
1008545995638,15cyclictest0-21swapper/312:15:593
100854299561,30cyclictest1090511-21taskset10:59:442
1008542995612,25cyclictest1137-21runrttasks09:37:082
100853799562,9cyclictest2069-21gnome-shell12:28:591
100853399569,7cyclictest13-21ksoftirqd/011:51:590
197399550,9rtkit-daemon1972-21rtkit-daemon07:39:440
197399550,9rtkit-daemon1972-21rtkit-daemon07:39:440
11231412550,5sleep0161rcuc/012:30:270
1008545995538,14cyclictest0-21swapper/311:33:593
1008545995538,14cyclictest0-21swapper/311:33:593
100854299551,50cyclictest1033-21gdbus07:52:352
100853799552,10cyclictest2069-21gnome-shell12:06:591
197399540,9rtkit-daemon1972-21rtkit-daemon09:45:370
197399540,8rtkit-daemon1972-21rtkit-daemon09:14:211
100854599542,48cyclictest2069-21gnome-shell10:41:593
1008542995436,15cyclictest0-21swapper/207:25:582
1008542995430,20cyclictest2069-21gnome-shell08:20:582
1008533995410,36cyclictest2069-21gnome-shell11:02:590
100854599532,10cyclictest2069-21gnome-shell09:47:593
1008545995316,11cyclictest1109116-21cat11:50:273
100853799532,10cyclictest2069-21gnome-shell11:35:591
100854599522,7cyclictest0-21swapper/308:27:583
100854599522,46cyclictest2069-21gnome-shell08:17:583
100854599522,10cyclictest2069-21gnome-shell10:33:593
100854299523,9cyclictest2069-21gnome-shell08:25:582
100854299522,46cyclictest1058469-40lsb_release09:26:042
100853799522,9cyclictest2069-21gnome-shell07:51:581
100853799522,9cyclictest2069-21gnome-shell07:35:581
100853799522,9cyclictest2069-21gnome-shell07:35:581
100853799522,46cyclictest2069-21gnome-shell10:55:591
100853399522,9cyclictest2069-21gnome-shell11:49:590
100853399522,45cyclictest2069-21gnome-shell12:16:590
100853399522,45cyclictest2069-21gnome-shell12:16:590
1008545995117,13cyclictest1042624-21systemd08:45:013
1008545995112,11cyclictest1018378-21kernelversion07:35:203
1008545995112,11cyclictest1018378-21kernelversion07:35:203
100854299512,46cyclictest0-21swapper/207:21:582
100854299512,45cyclictest2069-21gnome-shell07:58:582
100854299512,44cyclictest1041530-21hddtemp_smartct08:40:182
100854299511,46cyclictest1118279-40lsb_release12:17:032
100854299511,46cyclictest1118279-40lsb_release12:17:032
100854299511,45cyclictest1122405-21grep12:30:122
100854299511,45cyclictest1059189-21cpuspeed_turbos09:30:132
100854599502,8cyclictest0-21swapper/309:10:583
100854599502,8cyclictest0-21swapper/308:37:583
1008545995018,13cyclictest1043573-21ntpq08:45:233
1008545995011,13cyclictest0-21swapper/308:05:213
100854299502,45cyclictest2069-21gnome-shell10:10:592
100854299502,44cyclictest1040687-40lsb_release08:38:062
100854299502,44cyclictest0-21swapper/210:33:592
100854299502,42cyclictest1020048-21diskstats07:40:162
100854299501,45cyclictest1102728-21cat11:35:012
100854299501,45cyclictest1102728-21cat11:35:012
100854299501,44cyclictest1018156-21expr07:35:142
100854299501,44cyclictest1018156-21expr07:35:142
1008542995014,32cyclictest1054-21snmpd10:09:532
197399490,6rtkit-daemon1972-21rtkit-daemon08:15:431
10925452490,13sleep013-21ksoftirqd/011:05:110
100854599492,7cyclictest0-21swapper/310:51:593
100854599492,7cyclictest0-21swapper/310:51:593
100854599492,43cyclictest0-21swapper/310:21:593
100854599492,43cyclictest0-21swapper/310:00:593
100854599492,43cyclictest0-21swapper/310:00:593
100854599492,43cyclictest0-21swapper/309:34:593
100854299493,43cyclictest0-21swapper/211:37:592
100854299492,44cyclictest0-21swapper/208:04:582
100854299492,44cyclictest0-21swapper/208:04:582
100854299491,44cyclictest0-21swapper/211:40:122
100854299491,43cyclictest1085841-21cstates10:45:162
100854299491,43cyclictest1037553-21cpuspeed_turbos08:30:132
100854299491,42cyclictest1092632-21sort11:05:132
1008537994939,7cyclictest2069-21gnome-shell11:43:591
100853799491,6cyclictest4497-21PK-Backend08:24:581
100853399492,6cyclictest0-21swapper/011:57:590
100853399491,6cyclictest4497-21PK-Backend07:47:580
100854599489,12cyclictest1124537-21cat12:35:213
100854599482,6cyclictest0-21swapper/312:07:593
100854599482,6cyclictest0-21swapper/307:33:583
100854599482,43cyclictest42-21ksoftirqd/311:35:593
100854599482,43cyclictest0-21swapper/311:40:593
100854599482,42cyclictest0-21swapper/312:31:593
100854599482,42cyclictest0-21swapper/311:15:593
100854599482,42cyclictest0-21swapper/309:21:593
1008545994812,12cyclictest1106969-21missed_timers11:45:213
1008545994810,13cyclictest1118993-21grep12:20:133
1008545994810,12cyclictest1033-21gdbus10:15:013
100854299482,43cyclictest0-21swapper/210:19:592
1008542994817,26cyclictest1109966-21cpuspeed_turbos11:55:122
100854299481,44cyclictest1056426-21runrttasks09:20:442
100854299481,42cyclictest1088555-21chrt10:51:492
100854299481,42cyclictest1088555-21chrt10:51:492
1008537994840,5cyclictest1008527-21cyclictest08:39:581
100853799481,43cyclictest4497-21PK-Backend11:58:591
100853399483,38cyclictest2069-21gnome-shell12:39:590
1008533994833,11cyclictest2069-21gnome-shell10:53:590
1008533994833,11cyclictest2069-21gnome-shell10:53:590
100853399482,5cyclictest0-21swapper/010:06:590
100853399482,5cyclictest0-21swapper/008:00:580
100853399482,5cyclictest0-21swapper/008:00:580
100853399482,42cyclictest0-21swapper/007:22:580
100854599473,41cyclictest0-21swapper/310:58:593
100854599472,6cyclictest0-21swapper/308:52:583
100854599472,6cyclictest0-21swapper/308:33:583
100854599472,42cyclictest0-21swapper/312:11:593
100854599472,42cyclictest0-21swapper/312:11:593
100854599472,42cyclictest0-21swapper/311:58:593
100854599472,42cyclictest0-21swapper/311:22:593
100854599472,42cyclictest0-21swapper/311:14:593
100854599472,42cyclictest0-21swapper/308:56:583
100854599472,42cyclictest0-21swapper/307:42:583
1008545994713,11cyclictest1024446-21systemd-udevd07:52:353
100854299472,6cyclictest0-21swapper/211:53:582
100854299472,6cyclictest0-21swapper/211:28:582
100854299472,42cyclictest0-21swapper/207:33:582
100854299472,40cyclictest1407-21snapd10:20:132
100854299472,40cyclictest1043339-21irqstats08:45:202
100854299472,39cyclictest1106752-21df_inode11:45:152
1008542994716,27cyclictest1116295-21seq12:12:282
1008542994716,27cyclictest1116295-21seq12:12:272
100854299471,42cyclictest1114183-21gunzip12:05:312
100853799471,41cyclictest4497-21PK-Backend09:46:591
100853399472,5cyclictest0-21swapper/010:04:590
100853399472,5cyclictest0-21swapper/010:04:590
100853399471,5cyclictest4497-21PK-Backend09:53:590
100853399471,43cyclictest4497-21PK-Backend12:03:590
100853399471,43cyclictest4497-21PK-Backend07:56:580
100853399471,42cyclictest4497-21PK-Backend10:23:590
100854599462,24cyclictest1022199-21ntp_states07:45:243
1008545994619,9cyclictest1092143-21sshd11:03:303
1008545994614,7cyclictest1112624-21timerwakeupswit12:00:293
100854299462,41cyclictest0-21swapper/211:13:592
100854299462,39cyclictest1111879-21expr12:00:142
100854299461,41cyclictest1124740-21ntp_offset12:35:232
1008537994624,6cyclictest141rcu_preempt11:26:591
1008537994624,6cyclictest141rcu_preempt11:17:591
100853799462,42cyclictest0-21swapper/109:02:581
1008533994639,4cyclictest4497-21PK-Backend09:40:590
1008533994639,4cyclictest0-21swapper/008:13:580
100853399463,38cyclictest2069-21gnome-shell11:11:590
100853399461,5cyclictest4497-21PK-Backend09:32:590
100853799454,36cyclictest2069-21gnome-shell11:46:591
1008537994533,8cyclictest923042-21kworker/1:108:27:581
1008537994529,11cyclictest2069-21gnome-shell07:41:581
1008537994523,5cyclictest141rcu_preempt11:24:591
1008537994522,7cyclictest141rcu_preempt07:27:581
100854599441,22cyclictest1072564-21sshd10:05:303
100854299441,39cyclictest1022973-21cat07:50:012
100853799446,34cyclictest26-21ksoftirqd/108:48:581
1008537994437,5cyclictest231irq_work/111:09:591
100853799441,39cyclictest26-21ksoftirqd/112:30:591
100853799441,39cyclictest26-21ksoftirqd/112:03:591
100853799441,39cyclictest26-21ksoftirqd/107:48:581
1008533994437,4cyclictest4497-21PK-Backend12:08:580
1008533994436,4cyclictest4497-21PK-Backend11:32:590
1008533994436,4cyclictest4497-21PK-Backend11:32:590
10842602430,7chrt4497-21PK-Backend10:40:131
1008537994335,5cyclictest231irq_work/108:44:581
1008537994321,6cyclictest141rcu_preempt08:57:581
1008537994319,7cyclictest141rcu_preempt12:21:591
1008537994235,4cyclictest0-21swapper/109:15:591
1008537994233,6cyclictest231irq_work/110:51:591
1008537994233,6cyclictest231irq_work/110:51:591
100853799421,36cyclictest2069-21gnome-shell08:53:581
197399410,33rtkit-daemon4497-21PK-Backend08:06:160
1008537994135,4cyclictest0-21swapper/109:22:591
1008537994134,5cyclictest231irq_work/111:10:591
100853799413,34cyclictest26-21ksoftirqd/108:30:581
1008537994131,6cyclictest231irq_work/112:14:591
1008537994131,6cyclictest231irq_work/112:14:591
1008537994034,4cyclictest0-21swapper/109:50:591
100853399402,18cyclictest4497-21PK-Backend08:16:580
100853399402,11cyclictest1152-21snapd07:15:130
1008537993932,5cyclictest231irq_work/107:56:581
1008537993932,5cyclictest231irq_work/107:10:581
1008537993932,5cyclictest231irq_work/107:10:581
1008537993931,5cyclictest231irq_work/111:52:591
1008537993929,6cyclictest231irq_work/112:17:591
1008537993929,6cyclictest231irq_work/112:17:581
1008537993929,6cyclictest231irq_work/107:24:581
1008537993831,4cyclictest0-21swapper/110:11:591
1008533993815,6cyclictest141rcu_preempt09:10:200
1008537993730,5cyclictest231irq_work/111:04:591
1008537993728,6cyclictest231irq_work/108:12:581
100853799371,26cyclictest26-21ksoftirqd/112:36:591
1008533993725,8cyclictest4497-21PK-Backend08:33:580
1008533993720,13cyclictest1105239-21memory11:40:210
1008537993625,7cyclictest231irq_work/107:19:581
1008537993624,8cyclictest231irq_work/108:07:581
1008533993626,6cyclictest4497-21PK-Backend10:26:590
1008533993625,6cyclictest4497-21PK-Backend10:13:590
1008533993625,6cyclictest4497-21PK-Backend09:16:590
1008533993624,7cyclictest4497-21PK-Backend09:02:580
1008533993621,11cyclictest2069-21gnome-shell07:34:580
197399350,26rtkit-daemon1972-21rtkit-daemon09:35:140
1008537993526,6cyclictest0-21swapper/109:05:581
1008537993525,6cyclictest4497-21PK-Backend10:00:591
1008537993525,6cyclictest4497-21PK-Backend10:00:591
1008537993525,6cyclictest0-21swapper/110:37:591
1008533993527,5cyclictest0-21swapper/009:25:590
1008533993525,6cyclictest4497-21PK-Backend12:28:590
1008533993525,6cyclictest4497-21PK-Backend10:58:590
1008533993525,5cyclictest4497-21PK-Backend07:52:580
1008533993523,6cyclictest4497-21PK-Backend09:55:590
1008533993519,12cyclictest1009488-21ntpq07:10:200
1008533993519,12cyclictest1009488-21ntpq07:10:200
100853399351,29cyclictest4497-21PK-Backend08:39:580
1008537993425,6cyclictest0-21swapper/110:21:591
1008537993423,7cyclictest231irq_work/109:34:591
100853799341,30cyclictest0-21swapper/110:28:591
100853799341,29cyclictest4497-21PK-Backend10:30:591
100853799341,29cyclictest0-21swapper/109:28:231
1008533993423,6cyclictest4497-21PK-Backend10:36:590
1008533993422,7cyclictest4497-21PK-Backend08:25:580
1008533993419,11cyclictest1080961-21wc10:30:200
1008533993419,11cyclictest1055757-21awk09:20:200
1008533993418,12cyclictest2080-21gdbus09:06:460
100853799331,8cyclictest4497-21PK-Backend10:06:481
100853799331,8cyclictest4497-21PK-Backend08:02:131
100853799331,8cyclictest4497-21PK-Backend08:02:131
1008537993318,11cyclictest1063544-21sed09:40:281
100853399331,4cyclictest0-21swapper/010:16:000
100853399331,28cyclictest4497-21PK-Backend10:40:590
100853399331,28cyclictest4497-21PK-Backend08:45:190
1008537993222,6cyclictest4497-21PK-Backend09:38:591
1008533993019,7cyclictest1137-21runrttasks08:20:240
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional