| Commit message (Collapse) | Author | Age |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Debug features (DBFs) els_dbf, cmd_dbf and abt_dbf were removed and
san_dbf, hba_dbf and scsi_dbf were introduced. The erp_dbf did not
change.
The new traces improve debugging of problems with zfcp, scsi-stack,
multipath and hardware in the SAN. san_dbf traces things like ELS and
CT commands, hba_dbf saves HBA specific information of requests, and
scsi_dbf saves FCP and SCSI specific information of requests. Common
to all new DBFs is that they provide a so called structured view. This
significantly improves readability of the traces.
Signed-off-by: Andreas Herrmann <aherrman@de.ibm.com>
Signed-off-by: James Bottomley <James.Bottomley@SteelEye.com>
|
|
|
|
|
| |
Signed-off-by: Andreas Herrmann <aherrman@de.ibm.com>
Signed-off-by: James Bottomley <James.Bottomley@SteelEye.com>
|
|
|
|
|
| |
Signed-off-by: Andreas Herrmann <aherrman@de.ibm.com>
Signed-off-by: James Bottomley <James.Bottomley@SteelEye.com>
|
|
|
|
|
|
|
|
|
|
|
| |
o union zfcp_req_data removed
o increment unit refcount when processing FCP commands
(This fixes a theoretical race: When all scsi commands of a unit
are aborted and the scsi_device is removed then the unit could be
removed before all fsf_requests of that unit are completely processed.)
Signed-off-by: Andreas Herrmann <aherrman@de.ibm.com>
Signed-off-by: James Bottomley <James.Bottomley@SteelEye.com>
|
|
|
|
|
|
|
|
|
| |
o always use locking when changing erp_action lists,
o avoid escalation to ERP_ACTION_REOPEN_PORT_FORCED if erp_action is
still in use for ERP_ACTION_REOPEN_PORT
Signed-off-by: Andreas Herrmann <aherrman@de.ibm.com>
Signed-off-by: James Bottomley <James.Bottomley@SteelEye.com>
|
|
|
|
|
|
|
|
|
|
|
| |
On Thursday, September 15, 2005 6:22 AM, Christoph Hellwig wrote:
> Looks good to me, except for the spurious scsi_print_command prototype
> in mptscsih.h.
The attached patch addresses that concern.
Signed-off-by: Eric Moore <Eric.Moore@lsil.com>
Signed-off-by: James Bottomley <James.Bottomley@SteelEye.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Summary of Changes:
* splitting mpt_interrupt per Christophs suggestion
about a month ago
* rename ScsiCfgData to SpiCfgData structure,
then move all the raid related info into
new structure called RaidCfgData. This is
done because SAS supports RAID, as well as SPI,
so the raid stuff should be seperate.
* incorrect timeout calculation for cntdn
inside WaitForDoorbellAck and WaitForDoortbellInt
* add support for interpreting SAS Log Info
* Increase Event Log Size from 0xA to 0x32
* Fix bug in mptsas/mptfc/mptspi - when controller
has Initiator Mode Disabled, and only running in
TargetMode, the mptctl would panic when loading.
The fix is to return 0, instead of -ENODEV, in
SCSI LLD respective probe routines
* Fix bug in mptlan.c - driver will panic if
there is host reset, due to dev being set to
zero in mpt_lan_ioc_reset
* Fix's for SPI - Echo Buffer
* Several fix's in mptscsih_io_done - FCP Response
info, RESIDUAL_MISMATCH, Data Underrun, etc.
* Cleanup Error Handling - EH handlers,
mptscsih_flush_cmds, and zeroing out ScsiLookup
from mptscsih_qcmd
* Cleanup asyn event handling from
mptscsih -> mptscsih_event_process. Also
added support for SAS Persistent Table Full,
an asyn event
Signed-off-by: Eric Moore <Eric.Moore@lsil.com>
Signed-off-by: James Bottomley <James.Bottomley@SteelEye.com>
|
|
|
|
|
|
|
|
|
|
| |
Adds the actual mptsas driver, based upon the LSI driver with new work
for SAS transport class integration from Eric Moore and me.
This obviously depends on the SAS transport class.
Signed-off-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: James Bottomley <James.Bottomley@SteelEye.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
- various bits for SAS support from the LSI driver.
- use the device private data for the fusion target private data.
this should be using the midlayer target data framework, but we
can't move over to that until fusion has been switched to the
generic DV code
- use target ID and channel from the fusion target private data,
because those in scsi_device will be different for mptsas
Signed-off-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: James Bottomley <James.Bottomley@SteelEye.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
From: Alan Stern <stern@rowland.harvard.edu>
This patch (as561) fixes the error handler's thread-exit code. The
kthread_stop call won't wake the thread from a down_interruptible, so
the patch gets rid of the semaphore and simply does
set_current_state(TASK_INTERRUPTIBLE);
Signed-off-by: Alan Stern <stern@rowland.harvard.edu>
Modified to simplify the termination loop and correct the sleep condition.
Signed-off-by: James Bottomley <James.Bottomley@SteelEye.com>
|
|
|
|
|
|
|
|
|
| |
We fix the oops by enforcing the host state model. There have also
been two extra states added: SHOST_CANCEL_RECOVERY and
SHOST_DEL_RECOVERY so we can take the model through host removal while
the recovery thread is active.
Signed-off-by: James Bottomley <James.Bottomley@SteelEye.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This patch (as545) fixes the list traversals in __scsi_remove_target and
scsi_forget_host. In each case the existing code list_for_each_entry_safe
in an _unsafe_ manner, because the list was not protected from outside
modification while the iteration was running.
The new scsi_forget_host routine takes the moderately controversial step
of iterating over devices for removal rather than iterating over targets.
This makes more sense to me because the current scheme treats targets as
second-class citizens, created and removed on demand, rather than as
objects corresponding to actual hardware. (Also I couldn't figure out any
safe way to iterate over the target list, since it's not so easy to tell
when a target has already been removed.)
Signed-off-by: Alan Stern <stern@rowland.harvard.edu>
Signed-off-by: James Bottomley <James.Bottomley@SteelEye.com>
|
|
|
|
|
|
|
|
|
|
| |
I found one other thing that needs to be fixed. The call to
scsi_release_buffers in scsi_unprep_request causes an oops, because the
sgtable has already been freed in scsi_io_completion. The following patch
is needed.
Signed-off-by: Alan Stern <stern@rowland.harvard.edu>
Signed-off-by: James Bottomley <James.Bottomley@SteelEye.com>
|
|
|
|
|
| |
Signed-off-by: Alan Stern <stern@rowland.harvard.edu>
Signed-off-by: James Bottomley <James.Bottomley@SteelEye.com>
|
|
|
|
|
|
|
|
|
|
|
| |
From: Alan Cox <alan@redhat.com>
The virt_to_bus() wasn't correctly taken out of this driver. It needs
to be able to track both physical and virtual addresses for its prd table.
Update the driver to do this with separate tracking entries.
Signed-off-by: Alan Cox <alan@redhat.com>
Signed-off-by: James Bottomley <James.Bottomley@SteelEye.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
On Wed, 2005-09-14 at 18:06 +1000, Anton Blanchard wrote:
> And in particular it looks like the scsi_unprep_request in
> scsi_queue_insert is causing it. The following patch fixes the boot
> problems on the vscsi machine:
OK, my fault. Your fix is almost correct .. I was going to do this
eventually, honest, because there's no need to unprep and reprep a
command that comes in through scsi_queue_insert().
However, I decided to leave it in to exercise the scsi_unprep_request()
path just to make sure it was working. What's happening, I think, is
that we also use this path for retries. Since we kill and reget the
command each time, the retries decrement is never seen, so we're
retrying forever.
Signed-off-by: James Bottomley <James.Bottomley@SteelEye.com>
|
|
|
|
|
|
|
| |
Modules need a license to prevent kernel tainting.
Signed-off-by: Randy Dunlap <rdunlap@xenotime.net>
Signed-off-by: James Bottomley <James.Bottomley@SteelEye.com>
|
|
|
|
|
|
|
|
|
| |
This fixes an issue in scsi command initialization from a request
where sd, sr, st, and scsi_lib all fail to copy the request's
cmd_len to the scsi command's cmd_len field.
Signed-off-by: Timothy Thelin <timothy.thelin@wdc.com>
Signed-off-by: James Bottomley <James.Bottomley@SteelEye.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This patch moves aic7xxx over to the dma_get_required_mask() API and
dumps its open coded memory check.
It also appears from this bug:
https://bugzilla.redhat.com/bugzilla/show_bug.cgi?id=167049
That 39 bit addressing doesn't work on older cards. I surmise that the
AHC_LARGE_SCBS flag is the one that marks cards capable of using 39 bit
addressing, so I also folded that check into the code.
Signed-off-by: James Bottomley <James.Bottomley@SteelEye.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Linda Xie ever so gently pointed out that she had a patch
to preserve compatibility with older SLES targets, and I told
her we didn't need to push it to mainline.
This patch explicitly checks the version of the IBMVSCSI target
and ensures that large scatterlists are not sent to older
targets.
Signed-off-by: Linda Xie <lxie@us.ibm.com>
Signed-off-by: Dave Boutcher <boutcher@us.ibm.com>
Signed-off-by: James Bottomley <James.Bottomley@SteelEye.com>
|
|
|
|
|
|
|
| |
They report being SCSI-3 but seem to give back rubbish to a
REPORT_LUNS command. Force them to be sequentially scanned.
Signed-off-by: James Bottomley <James.Bottomley@SteelEye.com>
|
|\ |
|
| |
| |
| |
| |
| |
| |
| |
| |
| | |
set DID_NO_CONNECT for the BLKPREP_KILL case and correct a few
BLKPREP_DEFER cases that weren't checking for the need to plug the
queue.
Signed-Off-By: Mike Christie <michaelc@cs.wisc.edu>
Signed-off-by: James Bottomley <James.Bottomley@SteelEye.com>
|
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
The original API returned either an ERR_PTR() or a refcounted sdev.
Unfortunately, if it's successful, you need to do a scsi_device_put() on
the sdev otherwise the refcounting is wrong.
Everyone seems to expect that scsi_add_device() should be callable
without doing the ref put, so alter the API so it is (we still have
__scsi_add_device with the original behaviour).
The only actual caller that needs altering is the one in firewire ...
not because it gets this right, but because it acts on the error if one
is returned.
Acked-by: Stefan Richter <stefanr@s5r6.in-berlin.de>
Signed-off-by: James Bottomley <James.Bottomley@SteelEye.com>
|
| |
| |
| |
| |
| |
| |
| |
| |
| | |
This patch (as546) fixes an oops-causing failure to check the return code
from scsi_device_get. The call can return an error if the LLD is being
unloaded from memory.
Signed-off-by: Alan Stern <stern@rowland.harvard.edu>
Signed-off-by: James Bottomley <James.Bottomley@SteelEye.com>
|
| |
| |
| |
| |
| |
| | |
Signed-off-by: Andrew Vasquez <andrew.vasquez@qlogic.com>
Acked-by: Smart, James <James.Smart@emulex.com>
Signed-off-by: James Bottomley <James.Bottomley@SteelEye.com>
|
| |
| |
| |
| |
| | |
Signed-off-by: Andrew Vasquez <andrew.vasquez@qlogic.com>
Signed-off-by: James Bottomley <James.Bottomley@SteelEye.com>
|
| |
| |
| |
| | |
Signed-off-by: James Bottomley <James.Bottomley@SteelEye.com>
|
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
The attached patch updates the driver for the 3ware 9000 series to do
the following:
- Correctly handle single sgl's with use_sg = 1.
This is needed with the latest scsi-block-2.6 merge otherwise the 3w-9xxx
driver will not work. I tested the patch James sent a few weeks back to fix
this, and it had a bug where the request_buffer was accessed in
twa_scsiop_execute_scsi_complete() when it was invalid. This is a corrected
variation of that patch.
Signed-off-by: Adam Radford <linuxraid@amcc.com>
Signed-off-by: James Bottomley <James.Bottomley@SteelEye.com>
|
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
The SAS transport class contains common code to deal with SAS HBAs, an
aproximated representation of SAS topologies in the driver model,
and various sysfs attributes to expose these topologies and managment
interfaces to userspace.
In addition to the basic SCSI core objects this transport class introduces
two additional intermediate objects: The SAS PHY as represented by struct
sas_phy defines an "outgoing" PHY on a SAS HBA or Expander, and the SAS
remote PHY represented by struct sas_rphy defines an "incoming" PHY on a
SAS Expander or end device. Note that this is purely a software concept, the
underlying hardware for a PHY and a remote PHY is the exactly the same.
There is no concept of a SAS port in this code, users can see what PHYs
form a wide port based on the port_identifier attribute, which is the same
for all PHYs in a port.
This submission doesn't handle hot-plug addition or removal of SAS devices
and thus doesn't do scanning in a workqueue yet, that will be added in
phase2 after this submission. In a third phase I will add additional
managment infrastructure.
I think this submission is ready for 2.6.14, but additional comments are
of course very welcome.
I'd like to thanks James Smart a lot for his very useful input on the
design.
Signed-off-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: James Bottomley <James.Bottomley@SteelEye.com>
|
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
The soon to be released smartmontools 5.34 uses the
READ DEFECT DATA command on SCSI disks. A disk that
has defect list entries (or worse, an increasing number
of them) is at risk.
Currently the first invocation of smartctl causes this:
scsi: unknown opcode 0x37
message to appear the console and in the log.
The READ DEFECT DATA SCSI command does not change
the state of a disk. Its opcode (0x37) is valid for
SBC devices (e.g. disks) and SMC-2 devices (media
changers) where it is called INITIALIZE STATUS ELEMENT
WITH RANGE and again doesn't change the external state
of the device.
Changelog:
- mark SCSI opcode 0x37 (READ DEFECT DATA) as
safe_for_read
Signed-off-by: Douglas Gilbert <dougg@torque.net>
Acked-by: Jens Axboe <axboe@suse.de>
Signed-off-by: James Bottomley <James.Bottomley@SteelEye.com>
|
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
Further to the problem discussed in this post:
http://marc.theaimsgroup.com/?l=linux-scsi&m=112540053711489&w=2
It seems that the sg driver does not need to set the VM_IO flag
on pages that it memory maps to the user space since they are
not from the IO space. Ahmed Teirelbar <ahmed.teirelbar@adic.com>
wants the facility and has tested this patch as I have without
adverse effects.
The oops protection is still important. Some users really did
try and use dio transfers from the sg driver to memory mapped
IO space (on a video capture card if my memory serves) during the
lk 2.4 series. I'm not sure how successful it was but that will
now be politely refused in lk 2.6.13+ .
Changelog:
- set the page flags for sg's reserved buffer mmap-ed
to the user space to VM_RESERVED (rather than
VM_RESERVED | VM_IO )
Signed-off-by: Douglas Gilbert <dougg@torque.net>
Signed-off-by: James Bottomley <James.Bottomley@SteelEye.com>
|
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
Actually, just one problem and one cosmetic fix:
1) We need to dequeue for the loop and kill case (it seems easiest
simply to dequeue in the scsi_kill_request() routine)
2) There's no real need to drop the queue lock. __scsi_done() is lock
agnostic, so since there's no requirement, let's just leave it in to
avoid any locking issues.
Signed-off-by: James Bottomley <James.Bottomley@SteelEye.com>
|
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
From: Alan Stern <stern@rowland.harvard.edu>
This patch (as559b) adds a new routine, scsi_unprep_request, which
gets called every place a request is requeued. (That includes
scsi_queue_insert as well as scsi_requeue_command.) It also changes
scsi_kill_requests to make it call __scsi_done with result equal to
DID_NO_CONNECT << 16. (I'm not sure if it's necessary to call
scsi_init_cmd_errh here; maybe you can check on that.) Finally, the
patch changes the return value from scsi_end_request, to avoid
returning a stale pointer in the case where the request was requeued.
Fortunately the return value is used in only place, and the change
actually simplified it.
Signed-off-by: Alan Stern <stern@rowland.harvard.edu>
Rejections fixed up and
Signed-off-by: James Bottomley <James.Bottomley@SteelEye.com>
|
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
If a filesystem, while writing out data, decides that it is good
to issue a cache flush on a SCSI drive (or other 'sd' device), it will
call blkdev_issue_flush which calls ->issue_flush_fn which is
scsi_issue_flush_fn.
This calls sd_issue_flush which calls sd_sync_cache, which calls
scsi_execute_request.
This will (as sshdr != NULL) call
kmalloc(SCSI_SENSE_BUFFERSIZE, GFP_KERNEL)
If memory is tight, the presence of GFP_KERNEL may cause write
requests to be sent to some filesystem to free up memory, however if
that filesystem is waiting for the issue_flush_fn to complete, you
could get a deadlock.
I wonder if it might be more appropriate to use GFP_NOIO as in the
following patch.
I wonder if it might be even more appropriate to cope better with a
kmalloc failure, especially as in this use, sd_sync_cache only will
use the sense information to print out a more informative error
message.
Signed-off-by: Neil Brown <neilb@suse.de>
Signed-off-by: James Bottomley <James.Bottomley@SteelEye.com>
|
| |
| |
| |
| |
| |
| |
| |
| |
| | |
This patch (as544) adds a private entry point to scsi_remove_device, for
use when callers already own the scan_mutex. The appropriate callers are
modified to use the new entry point.
Signed-off-by: Alan Stern <stern@rowland.harvard.edu>
Signed-off-by: James Bottomley <James.Bottomley@SteelEye.com>
|
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
This patch (as543) adds a private entry point to scsi_scan_target, for use
when the caller already owns the scan_mutex, and updates the kerneldoc for
that routine (which was badly out-of-date). It converts scsi_scan_channel
to use the new entry point. Lastly, it modifies scsi_get_host_dev to make
it acquire the scan_mutex, necessary since the routine adds a new
scsi_device even if it doesn't do any actual scanning.
Signed-off-by: Alan Stern <stern@rowland.harvard.edu>
Signed-off-by: James Bottomley <James.Bottomley@SteelEye.com>
|
|\ \ |
|
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | | |
Use schedule_timeout_uninterruptible() instead of
set_current_state()/schedule_timeout() to reduce kernel size.
Signed-off-by: Nishanth Aravamudan <nacc@us.ibm.com>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: David S. Miller <davem@davemloft.net>
|
|\ \ \ |
|
| | | |
| | | |
| | | |
| | | |
| | | | |
Signed-off-by: John W. Linville <linville@tuxdriver.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
|
| | | |
| | | |
| | | |
| | | |
| | | | |
Signed-off-by: John W. Linville <linville@tuxdriver.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
|
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | | |
Instead, count them as part of rx_missed_errors.
Signed-off-by: John W. Linville <linville@tuxdriver.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
|
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | | |
Rename ax25_encapsulate to ax25_hard_header which these days more
accurately describes what the function is supposed to do.
Signed-off-by: Ralf Baechle DL5RB <ralf@linux-mips.org>
Signed-off-by: David S. Miller <davem@davemloft.net>
|
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | | |
Misc related cleanups in hamradio drivers:
o Use symbolic constants instead of magic numbers
o Don't try to handle the case where AX.25 isn't configured - the kernel
configuration doesn't permit that.
o Remove useless headers
Signed-off-by: Ralf Baechle DL5RB <ralf@linux-mips.org>
Signed-off-by: David S. Miller <davem@davemloft.net>
|
| |/ /
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | | |
Kernel connector - new userspace <-> kernel space easy to use
communication module which implements easy to use bidirectional
message bus using netlink as it's backend. Connector was created to
eliminate complex skb handling both in send and receive message bus
direction.
Connector driver adds possibility to connect various agents using as
one of it's backends netlink based network. One must register
callback and identifier. When driver receives special netlink message
with appropriate identifier, appropriate callback will be called.
From the userspace point of view it's quite straightforward:
socket();
bind();
send();
recv();
But if kernelspace want to use full power of such connections, driver
writer must create special sockets, must know about struct sk_buff
handling... Connector allows any kernelspace agents to use netlink
based networking for inter-process communication in a significantly
easier way:
int cn_add_callback(struct cb_id *id, char *name, void (*callback) (void *));
void cn_netlink_send(struct cn_msg *msg, u32 __groups, int gfp_mask);
struct cb_id
{
__u32 idx;
__u32 val;
};
idx and val are unique identifiers which must be registered in
connector.h for in-kernel usage. void (*callback) (void *) - is a
callback function which will be called when message with above idx.val
will be received by connector core.
Using connector completely hides low-level transport layer from it's
users.
Connector uses new netlink ability to have many groups in one socket.
[ Incorporating many cleanups and fixes by myself and
Andrew Morton -DaveM ]
Signed-off-by: Evgeniy Polyakov <johnpol@2ka.mipt.ru>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: David S. Miller <davem@davemloft.net>
|
|\ \ \ |
|
| |/ /
| | |
| | |
| | |
| | |
| | | |
Add platform independent parts of the ARM MPCore watchdog driver.
Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
|
|\ \ \ |
|
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | | |
The wbsd driver's card detection routing is a bit of a mess. This
patch cleans up the routine and makes it a bit more comprihensible.
Signed-off-by: Pierre Ossman <drzeus@drzeus.cx>
Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
|