You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-17 - 19:38

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackeslot3s.osadl.org (updated Sun May 17, 2026 12:45:26)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
154267699862,13cyclictest1567921-21munin-run09:50:000
154267699851,38cyclictest1632981-21ssh12:06:510
1542676998232,15cyclictest1619176-21ssh11:37:580
1542676998232,15cyclictest1619176-21ssh11:37:570
1542676998214,24cyclictest16850irq/134-800000011:05:010
154267699821,40cyclictest1551441-21ssh09:15:100
1542676997834,13cyclictest1-21systemd11:00:000
1542676997728,12cyclictest1587604-21ssh10:30:590
154267699771,13cyclictest1605430-21modprobe11:08:570
1542676997626,11cyclictest1640501-21ssh12:22:470
1542676997527,13cyclictest1593881-21ssh10:44:400
1542676997523,9cyclictest1569620-21ssh09:53:340
1542676997227,11cyclictest1613346-21ssh11:25:240
1542676997227,11cyclictest1613346-21ssh11:25:230
1542676997014,50cyclictest16850irq/134-800000011:41:040
154267699692,41cyclictest1582139-21cron10:20:000
1542676996626,18cyclictest12-21ksoftirqd/012:15:000
1542676996624,33cyclictest12-21ksoftirqd/012:05:030
1542676996519,12cyclictest1642454-21ssh12:27:000
1542676996517,12cyclictest1584984-21ssh10:25:260
1542676996417,13cyclictest1591354-21ssh10:39:230
1542676996325,17cyclictest1571607-21ssh09:57:380
154267699629,31cyclictest16850irq/134-800000010:01:240
1542676996232,12cyclictest12-21ksoftirqd/010:25:000
154267699608,30cyclictest16850irq/134-800000011:54:420
1542676996016,34cyclictest12-21ksoftirqd/011:20:010
1542676996016,34cyclictest12-21ksoftirqd/011:20:000
154267699593,3cyclictest131rcu_preempt08:55:010
1542676995920,19cyclictest12-21ksoftirqd/012:20:020
1542676995919,19cyclictest12-21ksoftirqd/012:40:000
1542676995826,13cyclictest1548931-21ssh09:10:010
1542681995740,13cyclictest1-21systemd10:00:001
1542676995727,14cyclictest1553855-21ssh09:20:130
1542676995624,10cyclictest1560597-21ssh09:34:470
154267699561,51cyclictest1598582-21ssh10:54:330
1542685995547,5cyclictest1546012-21kworker/2:309:00:012
1542685995538,8cyclictest1560736-21kworker/2:211:35:382
1542685995538,8cyclictest1560736-21kworker/2:211:35:382
1542676995525,15cyclictest1565264-21ssh09:44:300
1542676995525,15cyclictest1565264-21ssh09:44:290
1542685995446,5cyclictest1560736-21kworker/2:211:05:032
1542685995445,6cyclictest1543691-21kworker/2:010:40:012
154268199545,6cyclictest1553496-21cron09:20:001
1542681995438,13cyclictest1-21systemd08:25:011
1542676995427,12cyclictest1549298-21ssh09:10:470
1542676995426,13cyclictest1575939-21ssh10:06:360
154267699542,42cyclictest0-21swapper/010:15:000
1542685995344,6cyclictest1543691-21kworker/2:009:27:592
1542685995344,5cyclictest1543691-21kworker/2:009:59:042
154267699531,37cyclictest1644649-21ssh12:31:370
154269099525,3cyclictest131rcu_preempt12:22:443
1542690995231,14cyclictest3467602-21systemd-journal09:55:003
1542685995244,5cyclictest1543691-21kworker/2:009:40:062
1542685995244,5cyclictest1543691-21kworker/2:009:40:052
1542676995221,15cyclictest1557347-21ssh09:27:450
154269099514,4cyclictest131rcu_preempt11:31:333
1542685995135,8cyclictest1620019-21kworker/2:011:50:302
1542685995124,10cyclictest1560736-21kworker/2:211:08:562
154268199515,3cyclictest131rcu_preempt09:45:001
154268199515,3cyclictest131rcu_preempt09:45:001
1542681995126,5cyclictest23-21ksoftirqd/111:40:011
1542681995126,5cyclictest23-21ksoftirqd/111:40:001
154269099506,4cyclictest131rcu_preempt11:22:353
154269099505,4cyclictest131rcu_preempt12:09:503
154269099505,4cyclictest131rcu_preempt11:13:593
154269099505,4cyclictest131rcu_preempt09:22:493
154269099505,3cyclictest131rcu_preempt09:17:403
1542690995024,5cyclictest37-21ksoftirqd/310:35:003
154269099502,3cyclictest131rcu_preempt11:00:023
1542685995043,4cyclictest1560736-21kworker/2:211:31:562
1542685995042,5cyclictest1601232-21kworker/2:111:29:082
1542685995042,5cyclictest1601232-21kworker/2:111:29:082
1542685995042,5cyclictest1560736-21kworker/2:211:48:472
1542685995042,5cyclictest1543691-21kworker/2:010:00:092
1542685995042,5cyclictest1543691-21kworker/2:009:38:592
1542685995031,14cyclictest275-21dbus-daemon10:55:012
154268599502,3cyclictest131rcu_preempt09:55:022
1542685995012,6cyclictest131rcu_preempt10:30:152
154268199504,3cyclictest131rcu_preempt09:15:031
154268199503,3cyclictest131rcu_preempt10:30:011
1542681995023,5cyclictest131rcu_preempt12:20:021
154268199501,3cyclictest131rcu_preempt10:05:011
1542676995020,14cyclictest1607637-21ssh11:13:440
154269099496,4cyclictest131rcu_preempt09:47:363
1542690994933,12cyclictest1-21systemd11:40:003
1542690994933,12cyclictest1-21systemd11:40:003
1542685994941,5cyclictest1639072-21kworker/2:112:34:542
1542685994941,5cyclictest1560736-21kworker/2:211:12:052
1542685994941,5cyclictest1543691-21kworker/2:010:17:012
1542685994941,5cyclictest1543691-21kworker/2:008:50:022
1542685994941,5cyclictest1543691-21kworker/2:008:50:022
1542685994940,6cyclictest1546012-21kworker/2:309:06:572
1542685994940,6cyclictest1543691-21kworker/2:009:17:362
1542685994933,12cyclictest3467602-21systemd-journal09:05:012
1542685994925,8cyclictest1560736-21kworker/2:210:30:022
154268199497,3cyclictest150-21pr/ttyS210:22:311
154268199495,3cyclictest131rcu_preempt11:15:001
154268199494,3cyclictest131rcu_preempt12:10:011
154268199494,3cyclictest131rcu_preempt11:04:161
154268199493,3cyclictest131rcu_preempt11:30:011
154268199493,3cyclictest131rcu_preempt11:30:001
154268199492,4cyclictest131rcu_preempt10:20:011
154267699497,35cyclictest1542680-21latency_hist07:10:010
1542676994924,9cyclictest1615624-21ssh11:30:150
1542676994920,14cyclictest1561298-21ssh09:35:560
1542676994910,19cyclictest1611135-21ssh11:20:520
154269099485,4cyclictest131rcu_preempt09:32:153
154269099484,4cyclictest131rcu_preempt11:29:583
154269099484,4cyclictest131rcu_preempt11:29:573
154269099484,4cyclictest131rcu_preempt10:38:303
154269099483,3cyclictest131rcu_preempt12:00:123
1542685994840,5cyclictest1639072-21kworker/2:112:37:062
1542685994840,5cyclictest1620019-21kworker/2:012:25:452
1542685994840,5cyclictest1620019-21kworker/2:012:15:012
1542685994840,5cyclictest1560736-21kworker/2:211:58:242
1542685994840,5cyclictest1560736-21kworker/2:211:42:582
1542685994840,5cyclictest1543691-21kworker/2:010:57:052
1542685994840,5cyclictest1543691-21kworker/2:009:30:172
1542685994812,6cyclictest131rcu_preempt10:47:462
154268199485,3cyclictest131rcu_preempt11:15:441
154268199485,3cyclictest131rcu_preempt11:15:431
154268199484,4cyclictest131rcu_preempt10:51:561
154268199484,4cyclictest131rcu_preempt10:14:571
154268199484,3cyclictest131rcu_preempt09:54:371
154267699482,4cyclictest1-21systemd11:46:550
154267699481,40cyclictest0-21swapper/010:47:370
154269099476,3cyclictest131rcu_preempt10:43:203
154269099476,3cyclictest131rcu_preempt09:41:573
154269099476,3cyclictest131rcu_preempt09:41:563
154269099475,4cyclictest131rcu_preempt10:46:153
154269099475,4cyclictest131rcu_preempt10:03:043
154269099475,3cyclictest131rcu_preempt11:51:213
154269099474,4cyclictest131rcu_preempt12:20:023
154269099474,4cyclictest131rcu_preempt10:51:113
154269099474,4cyclictest131rcu_preempt10:20:103
154269099474,3cyclictest131rcu_preempt10:00:033
154269099474,3cyclictest131rcu_preempt08:05:003
154269099473,3cyclictest131rcu_preempt12:29:403
1542690994729,13cyclictest1636682-21munin-run12:15:013
1542690994723,5cyclictest37-21ksoftirqd/309:35:033
1542685994740,5cyclictest1543691-21kworker/2:009:23:502
1542685994739,5cyclictest1620019-21kworker/2:012:24:002
1542685994739,5cyclictest1620019-21kworker/2:012:00:102
1542685994739,5cyclictest1560736-21kworker/2:211:17:502
1542685994739,5cyclictest1560736-21kworker/2:211:17:492
1542685994739,5cyclictest1543691-21kworker/2:010:21:262
1542685994739,5cyclictest1535579-21kworker/2:307:25:092
1542685994726,10cyclictest30-21ksoftirqd/210:06:002
154268199477,8cyclictest23-21ksoftirqd/110:05:531
154268199475,3cyclictest131rcu_preempt12:40:011
154268199474,4cyclictest131rcu_preempt09:37:061
1542681994732,11cyclictest1-21systemd08:50:021
1542681994732,11cyclictest1-21systemd08:50:011
1542681994730,13cyclictest275-21dbus-daemon11:25:001
154268199472,3cyclictest131rcu_preempt09:00:011
1542681994713,4cyclictest131rcu_preempt12:00:501
1542676994740,4cyclictest16950irq/224-800000008:20:000
1542676994718,16cyclictest364-21snmpd07:25:020
154269099465,4cyclictest131rcu_preempt12:36:573
154269099465,3cyclictest131rcu_preempt10:28:183
154269099464,4cyclictest131rcu_preempt11:00:143
154269099464,4cyclictest131rcu_preempt10:07:093
154269099464,4cyclictest131rcu_preempt09:11:093
154269099464,3cyclictest131rcu_preempt11:05:233
154269099464,3cyclictest131rcu_preempt07:55:033
154269099464,3cyclictest131rcu_preempt07:55:023
154269099463,4cyclictest131rcu_preempt11:56:403
154269099462,3cyclictest131rcu_preempt09:27:343
154268599465,3cyclictest131rcu_preempt11:22:432
1542685994638,5cyclictest1543691-21kworker/2:010:44:062
1542685994638,5cyclictest1543691-21kworker/2:009:46:292
1542685994632,11cyclictest3467602-21systemd-journal10:12:322
154268199465,7cyclictest23-21ksoftirqd/112:34:371
154268199465,4cyclictest131rcu_preempt09:23:111
154268199465,3cyclictest131rcu_preempt11:51:111
154268199464,4cyclictest131rcu_preempt10:57:521
154268199464,4cyclictest131rcu_preempt10:45:391
154268199463,4cyclictest131rcu_preempt09:26:361
154268199463,3cyclictest131rcu_preempt11:30:221
1542681994610,6cyclictest131rcu_preempt08:30:011
154267699469,32cyclictest3467602-21systemd-journal07:45:000
154269099455,4cyclictest131rcu_preempt12:33:463
154269099455,3cyclictest131rcu_preempt10:19:473
154269099454,3cyclictest131rcu_preempt11:47:363
1542690994529,12cyclictest1-21systemd11:20:013
1542690994529,12cyclictest1-21systemd11:20:003
1542685994537,5cyclictest1620019-21kworker/2:012:08:552
154268599451,3cyclictest131rcu_preempt08:05:012
154268199455,4cyclictest131rcu_preempt10:43:071
154268199454,5cyclictest131rcu_preempt09:46:051
154268199453,8cyclictest23-21ksoftirqd/107:14:561
154268199452,4cyclictest131rcu_preempt12:22:471
154268199452,4cyclictest131rcu_preempt10:30:111
1542681994522,4cyclictest23-21ksoftirqd/108:45:001
154268199451,4cyclictest131rcu_preempt12:27:281
1542681994511,5cyclictest131rcu_preempt07:20:011
1542676994521,13cyclictest0-21swapper/008:30:010
1542690994421,7cyclictest37-21ksoftirqd/309:04:593
1542685994436,5cyclictest1543691-21kworker/2:008:40:042
154268199445,3cyclictest131rcu_preempt08:20:001
154268199444,3cyclictest131rcu_preempt11:57:511
154268199444,3cyclictest131rcu_preempt07:30:021
154268199443,4cyclictest131rcu_preempt09:31:341
154268199443,3cyclictest131rcu_preempt11:45:371
154268199443,3cyclictest131rcu_preempt11:45:021
1542681994432,9cyclictest23-21ksoftirqd/110:35:201
1542681994429,11cyclictest275-21dbus-daemon08:55:001
154268199442,3cyclictest131rcu_preempt12:11:491
154268199442,3cyclictest131rcu_preempt08:00:021
154268199441,3cyclictest131rcu_preempt11:10:001
154269099435,8cyclictest0-21swapper/308:49:563
154269099435,8cyclictest0-21swapper/308:49:553
154269099435,4cyclictest131rcu_preempt09:00:013
1542690994329,10cyclictest3467602-21systemd-journal07:15:013
1542690994327,12cyclictest3467602-21systemd-journal07:30:013
1542690994325,13cyclictest275-21dbus-daemon08:15:023
1542690994325,13cyclictest275-21dbus-daemon08:15:013
154268599439,5cyclictest131rcu_preempt08:16:552
154268599434,4cyclictest131rcu_preempt09:12:422
154268599434,4cyclictest131rcu_preempt08:10:022
1542685994335,5cyclictest1543691-21kworker/2:008:30:032
1542685994327,12cyclictest3467602-21systemd-journal12:20:012
1542685994323,15cyclictest275-21dbus-daemon08:25:002
154268199439,6cyclictest131rcu_preempt08:32:561
154268199432,3cyclictest131rcu_preempt08:10:001
154268199431,4cyclictest131rcu_preempt08:40:011
154268199431,4cyclictest131rcu_preempt08:05:021
154268199431,3cyclictest131rcu_preempt09:10:001
1542676994333,6cyclictest1543135-21sendmail07:20:000
1542676994330,9cyclictest12-21ksoftirqd/007:45:030
154267699431,38cyclictest1629150-21rm11:59:090
154269099425,3cyclictest131rcu_preempt11:41:133
154269099425,3cyclictest131rcu_preempt08:23:403
154269099424,3cyclictest131rcu_preempt10:11:493
154269099424,3cyclictest131rcu_preempt07:35:023
154269099424,3cyclictest131rcu_preempt07:35:023
154268599425,3cyclictest131rcu_preempt07:15:002
1542685994235,4cyclictest1535579-21kworker/2:307:19:412
154268599423,4cyclictest131rcu_preempt07:55:022
1542685994234,5cyclictest1543691-21kworker/2:008:10:032
1542685994234,5cyclictest1543691-21kworker/2:008:10:032
154268199424,4cyclictest131rcu_preempt08:15:021
154268199424,4cyclictest131rcu_preempt08:15:011
1542681994228,10cyclictest3467602-21systemd-journal07:35:011
1542681994228,10cyclictest3467602-21systemd-journal07:35:011
1542676994224,13cyclictest275-21dbus-daemon07:40:000
154269099416,8cyclictest37-21ksoftirqd/308:20:003
154269099415,8cyclictest37-21ksoftirqd/309:10:013
154269099414,3cyclictest131rcu_preempt08:41:563
154269099411,6cyclictest0-21swapper/308:30:573
154268599416,5cyclictest131rcu_preempt07:57:212
154268599416,5cyclictest131rcu_preempt07:57:202
154268199414,4cyclictest131rcu_preempt07:25:011
154268199414,3cyclictest131rcu_preempt07:55:021
154268199411,4cyclictest131rcu_preempt07:10:021
1542676994126,11cyclictest1548627-21sshd09:03:560
154267699411,6cyclictest1543670-21true07:30:000
1542690994024,12cyclictest3467602-21systemd-journal08:30:003
1542690994024,12cyclictest3467602-21systemd-journal07:35:033
154269099401,4cyclictest131rcu_preempt08:55:013
1542681994016,4cyclictest23-21ksoftirqd/107:50:021
154268199401,3cyclictest131rcu_preempt07:59:241
1542676994034,4cyclictest16950irq/224-800000008:08:570
1542676994026,11cyclictest1548014-21sed08:50:020
1542676994026,11cyclictest1548014-21sed08:50:020
1542676994023,14cyclictest1545170-21sshd07:56:570
1542676994023,14cyclictest1545170-21sshd07:56:560
154269099396,3cyclictest131rcu_preempt08:10:003
154269099392,8cyclictest37-21ksoftirqd/308:40:013
1542690993926,10cyclictest37-21ksoftirqd/307:45:003
1542690993921,14cyclictest275-21dbus-daemon07:20:013
154268599394,3cyclictest131rcu_preempt08:40:012
154268599394,3cyclictest131rcu_preempt07:43:552
154268599393,3cyclictest131rcu_preempt08:35:012
154268599393,3cyclictest131rcu_preempt07:35:022
154268599393,3cyclictest131rcu_preempt07:35:022
1542685993926,10cyclictest30-21ksoftirqd/207:40:012
154268599392,3cyclictest131rcu_preempt07:25:012
154267699393,31cyclictest0-21swapper/007:53:010
1542676993932,4cyclictest1547337-21latency_hist08:39:590
154268599381,4cyclictest131rcu_preempt08:55:002
154268199382,3cyclictest131rcu_preempt07:45:011
1542681993816,4cyclictest23-21ksoftirqd/109:00:021
154268199381,3cyclictest131rcu_preempt07:40:021
154267699382,5cyclictest1548413-21munin-run09:00:010
154267699382,5cyclictest1547081-21munin-run08:35:010
154269099373,3cyclictest131rcu_preempt07:50:013
154269099364,4cyclictest131rcu_preempt07:25:003
1542676993623,10cyclictest1542841-21sshd07:11:560
154267699362,30cyclictest1547613-21munin-plugin-st08:45:010
154267699361,31cyclictest0-21swapper/008:12:180
154267699361,31cyclictest0-21swapper/008:12:180
154269099355,3cyclictest131rcu_preempt07:52:563
154268599352,3cyclictest131rcu_preempt07:50:012
1542676993525,7cyclictest12-21ksoftirqd/007:35:030
1542676993525,7cyclictest12-21ksoftirqd/007:35:020
1542676993522,11cyclictest12-21ksoftirqd/008:24:580
154268599344,3cyclictest131rcu_preempt07:10:012
154267699342,3cyclictest131rcu_preempt08:05:010
154269099331,3cyclictest131rcu_preempt07:10:023
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional